Thursday, March 19, 2015

JavaFX on mobile, a dream come true!

Hi there!

It seems is time for a new post, but not a long and boring one as usual. I'll post briefly about my first experience bringing JavaFX to Google Play store.

Yes, I made it with 2048FX, a JavaFX version of the original game 2048, by Gabriel Cirulli, that Bruno Borges and I started last year.

Now, thanks to the outstanding work of JavaFXPorts project, I've adapted that version so it could be ported to Android. 

And with the very last version of their plugin, I've managed to succesfully submit it to Google Play store.

After a week in beta testing mode, today the app is in production, so you can go and download it to your Android device, and test it for yourself.

For those of you eager to get the app, this is the link. Go and get it, and add a nice review ;)

If you want to read about the process to make it possible, please keep on reading. These are the topics I'll cover in this post:
  • 2048FX, the game
  • JavaFXPorts and their mobile plugin
  • New Gluon plugin for NetBeans
  • 2048FX on Android
  • Google Play Store

2048FX, the game


Many of you will now for sure about the 2048 game by Gabriel Cirulli. Last year it was a hit.
Many of us got really addicted to it...


In case you are not one of those, the game is about moving numbers in a 4x4 grid,  and when equal numbers clash while moving the blocks (up/down/left/right), they merge and numbers are added up. The goal is reaching the 2048 tile (though you can keep going on looking for bigger ones!).

At that time, Bruno started a Java/JavaFX 8 version, given that the game was open sourced. I jumped in inmediately, and in a few weeks we had a nice JavaFX working version


Since we used (and learned) the great new features of Java 8, we thought it was a good proposal for JavaOne, and we end up presenting it in a talk (video) and doing a Hands on Lab session.

And we'll talk about it again next week at JavaLand. If you happen to be there, don't miss our talk

JavaFXPorts and their mobile plugin


Since the very beginning of JavaFX (2+), going mobile has been on the top of list of the most wanted features. We've dreamed with the possibility of making true the WORA slogan, and it's only recently since the appearance of the JavaFXPorts project, that this dream has come true.

Led by Johan Vos, he and his team have given to the community the missing piece, so now we can jump to mobile devices with the (almost) same projects we develop for desktop.

While Johan started this adventure at the end of 2013, his work on porting JavaFX to Android, based on the OpenJFX project, has been evolving constantly during 2014, until recently in Febrary 2015 he announced a join effort between his company, LodgOn, and Trillian Mobile, the company behind RoboVM, the open source project to port JavaFX to iOS.

As a result, jfxmobile-plugin, the one and only gradle JavaFX plugin for mobile was created and freely available through the JavaFXPorts repository.

With this plugin you can target from one single project three different platforms: Desktop, Android and iOS.

An it's as simple as this sample of build.gradle:
 
buildscript {
    repositories {
        jcenter()
    }
    dependencies {
        classpath 'org.javafxports:jfxmobile-plugin:1.0.0-b5'
    }
}

apply plugin: 'org.javafxports.jfxmobile'

repositories {
    jcenter()
}

mainClassName = 'org.javafxports.project.MainApplicationFX'

jfxmobile {
    ios {
        forceLinkClasses = [ 'org.javafxports.**.*' ]
    }
}

For Android devices, it is required Android SDK, and Android build-tools as you can read here. The rest of the depencencies (like Dalvik SDK and the Retrolambda plugin) are taking care by the plugin itself.

Note the plugin version 1.0.0-b5 is constantly evolving, and at the time of this writting 1.0.0-b7 is now available. Check this frequently to keep it updated.

With this plugin, several tasks are added to your project, and you can run any of them. Among others, these are the main ones:
  • ./gradlew android creates an Android package
  • ./gradlew androidInstall installs your Android application on an Android device that is connected to your development system via a USB cable.
  • ./gradlew launchIOSDevice launches your application on an iOS device that is connected to your development system
  • ./gradlew run launches your application on your development system.

New Gluon plugin for NetBeans


Setting up a complex project, with three different platforms can be a hard task. Until now,  the best approach (at least the one I followed) was cloning the HelloPlatform sample, and changing the project and package names.

But recently, a new company called Gluon, with Johan as one of his founding fathers,  has released a NetBeans plugin that extremelly simplifies this task.

Once you have installed the plugin, just create a JavaFX new project, and select Basic Gluon Application.



Select valid names for project, packages and main class, and you will find a bunch of folders in your new project:


Change the jfxmobile-plugin version to 1.0.0-b5 (or the most recent one), select one of the tasks mentioned before, and see for yourself.

 2048FX on Android


I've been using previous versions of the plugin, and it was a hard task to get everything working nicely. In fact, I had a working version of 2048FX on Android before the announcement of the jfxmobile-plugin. But it was a separated project from the desktop one.

Now with the plugin, everything binds together magically. So I have a single project with the desktop and the android version of the game.

Java 8?


There's a main drawback in all this process: Dalvik VM doesn't support Java 8 new features. For Lambdas, we can use the Retrolambda plugin, that takes care of converting them to Java 6 compatible bytecode. But Streams or Optional are not supported. This means that you have to manually backport them to Java 6/7 compatible version.

While the primary object of the 2048FX project was basically learning these features, for the sake of going mobile, I backported the project, though this didn't change its structure or its appearance.

The project: Desktop and Android altogether


This is how the project structure looks like:



A PlatformProvider interface allows us to find out in which platform we are running the project, which is extremely useful to isolate pieces of code that are natively related to that plaftorm.

For instance, to save the game session in a local file in the Android device, I need to access to an internal folder where the apk is installed, and for that I use an FXActivity instance, the bridge between JavaFX and Dalvik runtime, that extends Android Context.  This Context can be used to lookup Android services. 

One example of this is FileManager class, under Android packages:

import javafxports.android.FXActivity;
import android.content.Context;
import java.io.File;

public class FileManager {
    private final Context context;
    
    public FileManager(){
        context = FXActivity.getInstance();
    }
    
    public File getFile(String fileName){
        return new File(context.getFilesDir(), fileName);
    }
    
}

Now the PlatformProvider will call this implementation when running on Android, or the usual one for desktop.

After a few minor issues I had a working project in both desktop and Android.




Google Play Store


Bruno asked me once to go with this app to Google Play store, but at that time the project wasn't mature enough. But last weekend I decided to give it a try, so I enrolled myself in Google Play Developers, filled a form with the description and several screenshots, and finally submitted the apk of the game... what could go wrong, right?

Well, for starters, I had a first error: the apk had debug options enabled, and that was not allowed.

The AndroidManifest.xml


 This hidden file, created automatically by the plugin, contains important information of the apk. You can retrieve it after a first built, and modify it to include or modify different options. Then you have to refer this file in the build.gradle file.

On the application tag is where you have to add android:debuggable="false".

There you can add also the icon of your app: android:icon="@mipmap/ic_launcher", where mipmap-* are image folders with several resolutions.

Signing the apk


Well, that part was easy. Second attempt, second error... The apk must be signed for release. "Signed" means you need a private key, and for that we can use keytool.

And "release" means that we need to add to build.gradle the signing configuration... and that was not possible with the current plugin version b5.

So I asked Johan (on Friday night) about this, and he answered me (Saturday afternoon) that they've been working precisely on that but it was not ready yet. Later that evening, Joeri Sykora from LodgON, told me that it was in a branch... so with the invaluable help of John Sirach (the PiDome guy) we spent most of the Saturday night trying to build locally the plugin to add the signing configuration. 

It end up being something like this:

jfxmobile {
    android {
        signingConfig {
            storeFile file("path/to/my-release-key.keystore")
            storePassword 'STORE_PASSWORD'
            keyAlias 'KEY_ALIAS'
            keyPassword 'KEY_PASSWORD'
        }
        manifest = 'lib/android/AndroidManifest.xml'
        resDirectory = 'src/android/resources'
    }
}

It was done! It was almost 2 a.m., but I tried uploading the signed apk for the third time, and voilà!! No more errors. The app when to submission and in less than 10 hours, on Sunday morning it was already published!!

  

Beta Testing Program


Instead of going into production I chose the Beta Testing program, so during this week only a few guys have been able to access to Google Play to download and test the application.

Thanks to their feedback I've made a few upgrades, like fixing some issues with fonts and Samsung devices (thanks John) or changing the context menu to a visible toolbar (thanks Bruno).



2048FX on Google Play Store


And the beta testing time is over. As of now, the application is on production. 

What are you waiting for? Go and get it!! 

Download it, play with it, enjoy, and if you have any issue, any problem at all, please report it, so we can work on improving its usability in all kind of devices.

Final Thanks


Let me finish this post taking the word of the whole JavaFX community out there, saying out loud:


THANK YOU, JavaFXPorts !!!

Without you all of this wouldn't be possible at all.

Big thanks to all the guys already mentioned in this post, and also to Eugene Ryzhikov, Mark Heckler and Diego Cirujano, for helping along the way.

And finally, thanks to the OpenJFX project and the JavaFX team.

UPDATE

Thanks to the work of Jens Deters, 2048FX has made it to Apple Store too!



Go and install it from here!

And since today (15th May 2015), we are open sourcing all the project, so anyone can have a look at it and find out about the last final details required to put it all together and make it successfully to Google Play or Apple Store:

https://github.com/jperedadnr/Game2048FX

Enjoy!
 

Thursday, January 8, 2015

Creating and Texturing JavaFX 3D Shapes

Hi there!

It's been a while since my last post, and it seems I've just said the same on my last one... but you know, many stuff in between, and this blog has a tradition of long posts, those that can't be delivered on a weekly/monthly basis. But if you're a regular reader (thank you!) you know how this goes.

This post is about my last developments in JavaFX 3D, after working in close collaboration with a bunch of incredible guys for the last months.

For those of you new to this blog, I've already got a few posts talking about JavaFX 3D. My last recent one about the Rubik's Cube: RubikFX: Solving the Rubik's Cube with JavaFX 3D,  and other about the Leap Motion controller: Leap Motion Controller and JavaFX: A new touch-less approach.

I'll cover in this post the following topics:
Before getting started, did I mention my article "Building castles in the Sky. Use JavaFX 3D to model historical treasures and more" has been published in the current issue of Java Magazine?

In a nutshell,  this article describes a multi model JavaFX 3D based application, developed for the virtual immersion in Cultural Heritage Buildings, through models created by Reverse Engineering with Photogrammetry Techniques. The 3D model of the Menéndez Pelayo Library in Santander, Spain, is used throughout the article as an example of a complex model.



You can find this application and, thanks to Óscar Cosido, a free model of the Library here.

Leap Motion Skeletal Tracking Model


Since my first post about Leap Motion, I've improved the 3D version after Leap Motion released their  version 2 that includes an skeletal tracking model. 

I haven't had the chance to blog about it, but this early video shows my initial work. You can see that the model now includes bones, so a more realistic hand can be built.


I demoed a more advanced version at one of my JavaOne talks with the incredibles James Weaver, Sean Phillips and Zoran Sevarac. Sadly, Jason Pollastrini couldn't make it, but he was part of the 3D-team.

 

If you are interested, all the code is available here. Go, fork it and play with it if you have a Leap Motion controller.

 Yes, we did have a great time there.


The session was great. In fact you can watch it now at Parleys.

We had even a special guest: John Yoon, a.k.a. @JavaFX3D


Skinning Meshes and Leap Motion

And then I met Alexander Kouznetsov.

It was during the Hackergarten 3D session, where Sven Reimers and I were hacking some JavaFX 3D stuff, when he showed up, laptop in backpack, ready for some hacking. There's no better trick than asking a real developer:  I bet you're not able to hack this... to get it done!

So the challenge was importing a rigged hand in JSON format to use a SkinningMesh in combination with the new Leap Motion skeletal tracking model. As the one and only John Yoon would show later in his talk:
"In order to animate a 3D model, you need a transform hierarchy to which the 3D geometry is attached. 
The general term for this part of the pipeline is “rigging” or “character setup”.
Rigging is the process of setting up your static 3D model for computer-generated animation, to make it animatable."
He was in charge of animating the Duke for the chess demo shown at the Keynote of JavaOne 2013. As shown in the above picture, this required a mesh, a list of joints, weights and transformations, binding the inner 'bones' with the surrounding external mesh, so when the former were moved the latter was deformed, creating the desired animation effect.

The SkinningMesh class in the 3DViewer project was initially designed for Maya, and we had a rigged hand in Three.js model in JSON format.

So out of the blue Alex built an importer, and managed to get the mesh of the hand by reverse engineering. Right after that he solved the rest of the components of the skinningMesh. The most important part was the binding of the transformations between joints.


        Affine[] bindTransforms = new Affine[nJoints];
        Affine bindGlobalTransform = new Affine();
        List<Joint> joints = new ArrayList<>(nJoints);
        List<Parent> jointForest = new ArrayList<>();
        
        for (int i = 0; i < nJoints; i++) {
            JsonObject bone = object.getJsonArray("bones").getJsonObject(i);
            Joint joint = new Joint();
            String name = bone.getString("name");
            joint.setId(name);
            JsonArray pos = bone.getJsonArray("pos");
            double x = pos.getJsonNumber(0).doubleValue();
            double y = pos.getJsonNumber(1).doubleValue();
            double z = pos.getJsonNumber(2).doubleValue();
            joint.t.setX(x);
            joint.t.setY(y);
            joint.t.setZ(z);
            bindTransforms[i] = new Affine();
            int parentIndex = bone.getInt("parent");
            if (parentIndex == -1) {
                jointForest.add(joint);
                bindTransforms[i] = new Affine(new Translate(-x, -y, -z));
            } else {
                Joint parent = joints.get(parentIndex);
                parent.getChildren().add(joint);
                bindTransforms[i] = new Affine(new Translate(
                        -x - parent.getLocalToSceneTransform().getTx(), 
                        -y - parent.getLocalToSceneTransform().getTy(), 
                        -z - parent.getLocalToSceneTransform().getTz()));
            }
            joints.add(joint);
            joint.getChildren().add(new Axes(0.02));
        }

This was the first animation with the model:

 

The axes are shown at every joint. Observe how easy is to deform a complex mesh just by rotating two joints:

       Timeline t = new Timeline(new KeyFrame(Duration.seconds(1), 
                new KeyValue(joints.get(5).rx.angleProperty(), 90),
                new KeyValue(joints.get(6).rx.angleProperty(), 90)));
        t.setCycleCount(Timeline.INDEFINITE);
        t.play(); 

With a working SkinningMesh, it was just time for adding the skeletal tracking model from Leap Motion. 

First, we needed to match Bones to joints, and then we just needed to apply the actual orientation of every bone to the corresponding joint transformation.


        listener = new LeapListener();
        listener.doneLeftProperty().addListener((ov,b,b1)->{
            if(b1){
                List<finger$gt; fingersLeft=listener.getFingersLeft();
                Platform.runLater(()->{
                    fingersLeft.stream()
                        .filter(finger -> finger.isValid())
                        .forEach(finger -> {
                            previousBone=null;
                            Stream.of(Bone.Type.values()).map(finger::bone)
                                .filter(bone -> bone.isValid() && bone.length()>0)
                                .forEach(bone -> {
                                    if(previousBone!=null){
                                        Joint joint = getJoint(false,finger,bone);
                                        Vector cross = bone.direction().cross(previousBone.direction());
                                        double angle = bone.direction().angleTo(previousBone.direction());
                                        joint.rx.setAngle(Math.toDegrees(angle));
                                        joint.rx.setAxis(new Point3D(cross.getX(),-cross.getY(),cross.getZ()));
                                    }
                                    previousBone=bone;
                            });
                    });
                    ((SkinningMesh)skinningLeft.getMesh()).update();
                });
            }
        });

The work was almost done! Back from JavaOne I had the time to finish the model, adding hand movements and drawing the joints:



This video sums up most of what we've accomplished:


If you are interested in this project, all the code is here. Feel free to clone or fork it. Pull requests will be very wellcome.

 TweetWallFX

One thing leads to another... And Johan Vos and Sven asked me to join them in a project to create a Tweet Wall with JavaFX 3D for Devoxx 2014. JavaFX 3D? I couldn't say no even if I wasn't attending!

Our first proposal (not the one Sven finally accomplished) was based on the F(X)yz library from Sean and Jason: a SkyBox as a container, with several tori inside, where tweets were rotating over them:


Needless to say, we used the great Twitter4J API for retrieving new tweets with the hashtag #Devoxx.

The first challenge here was figuring out how to render the tweets over each torus. The solution was based on the use of an snapshot of the tweet (rendered in a background scene) that would serve as the diffuse map image of the PhongMaterial assigned to the torus.

To second was creating a banner effect rotating the tweets over they tori. To avoid artifacts, a segmented torus was built on top of the first one,  cropping the faces of a regular torus, so the resulting mesh will be textured with the image.

This is our desired segmented torus. 



In the next section, we'll go into details of how we could accomplish this shape.

Creating new 3D shapes

Note to beginners: For an excelent introduction to  JavaFX 3D, have a look to the 3D chapters on these books: JavaFX 8 Introduction by Example an JavaFX 8 Pro: A Definitive Guide to Building Desktop, Mobile, and Embedded Java Clients.

To create this mesh in JavaFX 3D we use a TriangleMesh as a basis for our mesh, where we need to provide float arrays of vertices and texture coordinates and one int array of vertex and texture indices for defining every triangle face.

Since a torus can be constructed from a rectangle, by gluting both pairs of opposite edges together with no twists, we could use a 2D rectangular grid in a local system ($\theta$,$\phi$), and map every point with these equations:

\[X=(R+r \cos\phi) \cos\theta\\Z=(R+r \cos\phi) \sin\theta\\Y=r \sin\phi\]

So based on this grid (with colored borders and triangles for clarity):

 we could create this torus (observe how the four corners of the rectangle are joinned together in one single vertex):


Now if we want to segment the mesh, we can get rid of a few elements from the borders. From the red inner grid, we could have a segmented torus now:



Vertices coordinates

As we can see in the SegmentedTorusMesh class from the F(x)yz library, generating the vertices for the mesh is really easy, based in the above equations, the desired number of subdivisions (20 and 16 in the figures) and the number of elements cropped in both directions (4):

       
    private TriangleMesh createTorus(int subDivX, int subDivY, int crop, float R, float r){    
        TriangleMesh triangleMesh = new TriangleMesh();

        // Create points
        List<Point3D> listVertices = new ArrayList<>();
        float pointX, pointY, pointZ;
        for (int y = crop; y <= subDivY-crop; y++) {
            float dy = (float) y / subDivY;
            for (int x = crop; x <= subDivX-crop; x++) {
                float dx = (float) x / subDivX;
                if(crop>0 || (crop==0 && x<subDivX && y<subDivY)){
                    pointX = (float) ((R+r*Math.cos((-1d+2d*dy)*Math.PI))*Math.cos((-1d+2d*dx)*Math.PI));
                    pointZ = (float) ((R+r*Math.cos((-1d+2d*dy)*Math.PI))*Math.sin((-1d+2d*dx)*Math.PI));
                    pointY = (float) (r*Math.sin((-1d+2d*dy)*Math.PI));
                    listVertices.add(new Point3D(pointX, pointY, pointZ));
                }
            }
        }

Note that we have to convert this collection to a float array. Since there is no such thing as FloatStream, trying to use Java 8 streams, I asked a question at StackOverflow, and as result now we use a very handy FloatCollector to do the conversion:

        float[] floatVertices=listVertices.stream()
            .flatMapToDouble(p->new DoubleStream(p.x,p.y,p.z))
            .collect(()->new FloatCollector(listVertices.size()*3), FloatCollector::add, FloatCollector::join)
            .toArray();

        triangleMesh.getPoints().setAll(floatVertices);

In case anybody is wondering why we don't use plain float[], using collections instead of simple float arrays allow us to perform mesh coloring (as we'll see later), subdivisions, ray tracing,...using streams and, in many of these cases, parallel streams.

Well, in Jason's words: why TriangleMesh doesn't provide a format that incorporates the use of streams by default...??

Texture coordinates

In the same way, we can create the texture coordinates. We can use the same grid, but now mapping (u,v) coordinates, from (0.0,0.0) on the left top corner to (1.0,1.0) on the right bottom one.


We need extra points for the borders.

        int index=0;
        int width=subDivX-2*crop;
        int height=subDivY-2*crop;
        float[] textureCoords = new float[(width+1)*(height+1)*2];
        for (int v = 0; v <= height; v++) {
            float dv = (float) v / ((float)(height));
            for (int u = 0; u <= width; u++) {
                textureCoords[index] = (float) u /((float)(width));
                textureCoords[index + 1] = dv;
                index+=2;
            }
        }
        triangleMesh.getTexCoords().setAll(textureCoords);

Faces

Once we have defined the coordinates we need to create the faces. From JavaDoc:
The term face is used to indicate 3 set of interleaving points and texture coordinates that together represent the geometric topology of a single triangle.
One face is defined by 6 indices: p0, t0, p1, t1, p2, t2, where p0, p1 and p2 are indices into the points array, and t0, t1 and t2 are indices into the texture coordinates array.

For convenience, we'll use two splitted collections of points indices and texture indices.

Based on the above figures, we go triangle by triangle, selecting the three indices position in specific order. This is critical for the surface orientation. Also note that for vertices we reuse indices at the borders to avoid the formation of seams.

        List<Point3D> listFaces = new ArrayList<>();
        // Create vertices indices
        for (int y =crop; y<subDivY-crop; y++) {
            for (int x=crop; x<subDivX-crop; x++) {
                int p00 = (y-crop)*((crop>0)?numDivX:numDivX-1) + (x-crop);
                int p01 = p00 + 1;
                if(crop==0 && x==subDivX-1){
                    p01-=subDivX;
                }
                int p10 = p00 + ((crop>0)?numDivX:numDivX-1);
                if(crop==0 && y==subDivY-1){
                    p10-=subDivY*((crop>0)?numDivX:numDivX-1);
                }
                int p11 = p10 + 1;
                if(crop==0 && x==subDivX-1){
                    p11-=subDivX;
                }                
                listFaces.add(new Point3D(p00,p10,p11));                
                listFaces.add(new Point3D(p11,p01,p00));
            }
        }

        List<Point3D> listTextures = new ArrayList<>();
        // Create textures indices
        for (int y=crop; y<subDivY-crop; y++) {
            for (int x=crop; <subDivX-crop; x++) {
                int p00 = (y-crop) * numDivX + (x-crop);
                int p01 = p00 + 1;
                int p10 = p00 + numDivX;
                int p11 = p10 + 1;
                listTextures.add(new Point3D(p00,p10,p11));                
                listTextures.add(new Point3D(p11,p01,p00));
            }
        }
       
Though now we have to join them. The adventages of this approach will be shown later.

        // create faces
        AtomicInteger count=new AtomicInteger();
        int faces[] = return listFaces.stream()
            .map(f->{
                Point3D t=listTexture.get(count.getAndIncrement());
                int p0=(int)f.x; int p1=(int)f.y; int p2=(int)f.z;
                int t0=(int)t.x; int t1=(int)t.y; int t2=(int)t.z;
                return IntStream.of(p0, t0, p1, t1, p2, t2);
            }).flatMapToInt(i->i).toArray();
        triangleMesh.getFaces().setAll(faces);
    
        // finally return mesh
        return triangleMesh;
    }

This picture shows how we create the first and last pairs of faces. Note the use of counterclockwise winding to define the front faces, so we have the normal of every surface pointing outwards (to the outside of the screen).


Finally, we can create our banner effect, adding two tori, both solid (DrawMode.FILL) and one of them segmented and textured with an image. This snippet shows the basics:

        SegmentedTorusMesh torus = new SegmentedTorusMesh(50, 42, 0, 500d, 300d); 
        PhongMaterial matTorus = new PhongMaterial(Color.FIREBRICK);
        torus.setMaterial(matTorus);
        
        SegmentedTorusMesh banner = new SegmentedTorusMesh(50, 42, 14, 500d, 300d); 
        PhongMaterial matBanner = new PhongMaterial();
        matBanner.setDiffuseMap(new Image(getClass().getResource("res/Duke3DprogressionSmall.jpg").toExternalForm()));
        banner.setMaterial(matBanner); 
     
        Rotate rotateY = new Rotate(0, 0, 0, 0, Rotate.Y_AXIS);
        torus.getTransforms().addAll(new Rotate(0,Rotate.X_AXIS),rotateY);
        banner.getTransforms().addAll(new Rotate(0,Rotate.X_AXIS),rotateY);
   
        Group group.getChildren().addAll(torus,banner);        
        Group sceneRoot = new Group(group);
        Scene scene = new Scene(sceneRoot, 600, 400, true, SceneAntialiasing.BALANCED);
        primaryStage.setTitle("F(X)yz - Segmented Torus");
        primaryStage.setScene(scene);
        primaryStage.show(); 

        final Timeline bannerEffect = new Timeline();
        bannerEffect.setCycleCount(Timeline.INDEFINITE);
        final KeyValue kv1 = new KeyValue(rotateY.angleProperty(), 360);
        final KeyFrame kf1 = new KeyFrame(Duration.millis(10000), kv1);
        bannerEffect.getKeyFrames().addAll(kf1);
        bannerEffect.play();

to get this animation working:

 

Playing with textures

The last section of this long post will show you how we can hack the textures from a TriangleMesh to display more advances images over the 3D shape. This will include:
  • Coloring meshes (vertices or faces) 
  • Creating contour plots
  • Using patterns
  • Animating textures
This work is inspired by a question from Álvaro Álvarez on StackOverflow, about coloring individual triangles or individual vertices from a mesh. The inmediate answer would be: no, you can't easily, since for one mesh there's one material with one diffuse color, and it's not possible to assing different materials to different triangles of the same mesh. You could create as many meshes and materials as colors, if this number were really small.

Using textures, was the only way, but for that, following the standard procedure, you will need to color precisely your texture image, to match each triangle with each color. 

In convex polihedra there's at least one net, a 2D arrangement of polygons that can be folded into the faces of the 3D shape. Based on an icosahedron (20 faces), we could use its net to color every face:



And then use the image as texture for the 3D shape:


This was my first answer, but I started thinking about using another approach. What if instead of the above colored net we could create on runtime a small image of colored rectangles, like this:


and trick the texture coordinates and texture indices to find their values in this image instead? Done! The result was this more neat picture:



(The required code to do this is in my answer, so I won't post it here). 

And going a little bit further, if we could create one palette image, with one color per pixel, we could also assign one color to each vertex, and the texture for the rest of the triangle will be interpolated by the scene graph! This was part of a second answer:


Color Palette

With this small class we can create small images with up to 1530 unique colors. The most important thing is they are correlative, so we'll have smooth contour-plots, and there won't be unwanted bumps when intermediate values are interpolated.



To generate on runtime this 40x40 image (2 KB) we just use this short snippet:

        Image imgPalette = new WritableImage(40, 40);
        PixelWriter pw = ((WritableImage)imgPalette).getPixelWriter();
        AtomicInteger count = new AtomicInteger();
        IntStream.range(0, 40).boxed()
                .forEach(y->IntStream.range(0, 40).boxed()
                        .forEach(x->pw.setColor(x, y, Color.hsb(count.getAndIncrement()/1600*360,1,1))));

With it, we can retrieve the texture coordinates for a given point from this image and update the texture coordinates on the mesh:

    public DoubleStream getTextureLocation(int iPoint){
        int y = iPoint/40; 
        int x = iPoint-40*y;
        return DoubleStream.of((((float)x)/40f),(((float)y)/40f));
    }

    public float[] getTexturePaletteArray(){
        return IntStream.range(0,colors).boxed()
            .flatMapToDouble(palette::getTextureLocation)
            .collect(()->new FloatCollector(2*colors), FloatCollector::add, FloatCollector::join)
            .toArray();
    }

    mesh.getTexCoords().setAll(getTexturePaletteArray());

Density Maps

Half of the work is done. The other half consists in assigning a color to every vertex or face in our mesh, based on some criteria. By using a mathematical function that for any $(x,y,z)$ coordinates we'll have a value $f(x,y,z)$ that can be scaled within our range of colors.

So let's have a function:

    @FunctionalInterface
    public interface DensityFunction<T> {
        Double eval(T p);
    }

    private DensityFunction<Point3D> density;

Let's find the extreme values, by evaluating the given function in all the vertices, using parallel streams:

    private double min, max;

    public void updateExtremes(List<Point3D> points){
        max=points.parallelStream().mapToDouble(density::eval).max().orElse(1.0);
        min=points.parallelStream().mapToDouble(density::eval).min().orElse(0.0);
        if(max==min){
            max=1.0+min;
        }
    }

Finally, we assign the color to every vertex in every face, by evaluating the given function in all the vertices, using parallel streams:
    
    public int mapDensity(Point3D p){
        int f=(int)((density.eval(p)-min)/(max-min)*colors);
        if(f<0){
            f=0;
        }
        if(f>=colors){
            f=colors-1;
        }
        return f;
    }

    public int[] updateFacesWithDensityMap(List<Point3D> points, List<Point3D> faces){
        return faces.parallelStream().map(f->{
                int p0=(int)f.x; int p1=(int)f.y; int p2=(int)f.z;
                int t0=mapDensity(points.get(p0));
                int t1=mapDensity(points.get(p1));
                int t2=mapDensity(points.get(p2));
                return IntStream.of(p0, t0, p1, t1, p2, t2);
            }).flatMapToInt(i->i).toArray();
    }

    mesh.getFaces().setAll(updateFacesWithDensityMap(listVertices, listFaces));

Did I say I love Java 8??? You can see now how the strategy of using lists for vertices, textures and faces has clear adventages over the float arrays.

Let's run some example, using the IcosahedronMesh classs from F(X)yz:
    
    IcosahedronMesh ico = new IcosahedronMesh(5,1f);
    ico.setTextureModeVertices3D(1600,p->(double)p.x*p.y*p.z);
    Scene scene = new Scene(new Group(ico), 600, 600, true, SceneAntialiasing.BALANCED);
    primaryStage.setScene(scene);
    primaryStage.show();     

This is the result:


Impressive, right? After a long explanation, we can happily say: yes! we can color every single triangle or vertex on the mesh!

And we could even move the colors, creating an smooth animation. For this we only need to update the faces (vertices and texture coordinates are the same). This video shows one:


More features

More? In this post? No! I won't extend it anymore. I just post this picture:




And refer you to all these available 3D shapes and more at F(X)yz repository. If I have the time, I'll try to post about them in a second part.

Conclusions

JavaFX 3D API in combination with Java 8 new features has proven really powerful in terms of rendering complex meshes. The API can be easily extended to create libraries or frameworks that help the developer in case 3D features are required.

We are  far from others (Unity 3D, Three.js, ... to say a few), but with the collaboration of the great JavaFX community we can shorten this gap.

Please, clone the repository, test it, create pull requests, issues, feature requests, ... get in touch with us, help us to keep this project alive and growing.

Also visit StackOverflow and ask questions there using these tags: javafx, javafx-8 and the new javafx-3d). You never know where a good question may take you! And the answers will help others developers too.

A final word to give a proper shout-out to Sean Phillips and Jason Pollastrini, founders of the F(x)yz library, for starting an outstanding project.


Monday, April 14, 2014

RubikFX: Solving the Rubik's Cube with JavaFX 3D

Hi all!

It's really been a while since my last post... But in the middle, three major conferences kept me away from my blog. I said to myself I had to blog about the topics of my talks, but I had no chance at all. 

Now, with JavaOne still far in the distance, and after finishing my collaboration in the book, soon to be published, JavaFX 8, Introduction by Example, with my friends Carl Dea, Gerrit Grunwald, Mark Heckler and Sean Phillips, I've had the chance to play with Java 8 and JavaFX 3D for a few weeks, and this post is the result of my findings.

It happens that my kids had recently at home a Rubik's cube, and we also built the Lego Mindstorms EV3 solver thanks to this incredible project from David Gilday, the guy behind the CubeStormer 3 with the world record of fastest solving.




After playing with the cube for a while, I thought about the possibility of creating a JavaFX application for solving the cube, and that's how RubikFX was born.

If you're eager to know what this is about, here is a link to a video in YouTube which will show you most of it. 

Basically, in this post I'll talk about importing 3D models in a JavaFX scene, with the ability to zoom, rotate, scale them, add lights, move the camera,... Once we have a nice 3D model of the Rubik's cube, we will try to create a way of moving layers independently from the rest of the model, keeping track of the changes made. Finally, we'll add mouse picking for selecting faces and rotating layers.

Please read this if you're not familiar with Rubik's cube notation and the basic steps for solving it.

Before we start

By now you should know that Java 8 is GA since the 18th of March, so the code in this project is based on this version. In case you haven't done it yet, please download from here the new SDK and update your system. Also, I use NetBeans 8.0 for its support for Java 8 including lambdas and the new Streams API, among other things. You can update your IDE from here.

I use two external dependencies. One for importing the model, from a experimental project called 3DViewer, which is part of the OpenJFX project. So we need to download it and build it. The second one is from the ControlsFX project, for adding cool dialogs to the application. Download it from here.

Finally, we need a 3D model for the cube. You can build it yourself or use a free model, like this, submitted by 3dregenerator, which you can download in 3ds or OBJ formats.

Once you've got all this ingredients, it's easy to get this picture:



For that, just extract the files, rename 'Rubik's Cube.mtl' to 'Cube.mtl' and 'Rubik's Cube.obj' to 'Cube.obj', edit this file and change the third line to 'mtllib Cube.mtl', and save the file.

Now run the 3DViewer application, and drag 'Cube.obj' to the viewer. Open Settings tab, and select Lights, turn on the ambient light with a white color, and off the puntual Light 1. You can zoom in or out (mouse wheel, right button or navigation bar), rotate the cube with the left button (modifying rotation speed with Ctrl or Shift), or translate the model with both mouse buttons pressed.

Now select Options and click Wireframe, so you can see the triangle meshes used to build the model.




Each one of the 27 cubies is given a name in the obj file, like 'Block46' and so on. All of its triangles are grouped in meshes defined by the material assigned, so each cubie is made of 1 to 6 meshes, with names like 'Block46', 'Block46 (2)' and so on, and there are a total number of 117 meshes.

The color of each cubie meshes is asigned in the 'Cube.mtl' file with the Kd constant relative to the diffuse color. 

1. The Rubik's Cube - Lite Version

Importing the 3D model

So once we know how is our model, we need to construct the MeshView nodes for each mesh. The ObjImporter class from 3DViewer provide a getMeshes() method that returns a Set of names of the blocks for every mesh. So we will define a HashMap to bind every name to its MeshView. For each mesh name s we get the MeshView object with buildMeshView(s) method. 

By design, the cube materials in this model don't reflect light (Ns=0), so we'll change this to allow interaction with puntual lights, by modifying the material specular power property, defined in the PhongMaterial class.

Finally, we will rotate the original model so we have the white face on the top, and the blue one on the front.

public class Model3D {
    
    // Cube.obj contains 117 meshes, marked as "Block46",...,"Block72 (6)" in this set: 
    private Set<String> meshes;

    // HashMap to store a MeshView of each mesh with its key
    private final Map<String,Meshview> mapMeshes=new HashMap<>();
    
    public void importObj(){
        try {// cube.obj
            ObjImporter reader = new ObjImporter(getClass().getResource("Cube.obj").toExternalForm());
            meshes=reader.getMeshes(); // set with the names of 117 meshes
                      
            Affine affineIni=new Affine();            
            affineIni.prepend(new Rotate(-90, Rotate.X_AXIS));
            affineIni.prepend(new Rotate(90, Rotate.Z_AXIS));
            meshes.stream().forEach(s-> { 
                MeshView cubiePart = reader.buildMeshView(s);
                // every part of the cubie is transformed with both rotations:
                cubiePart.getTransforms().add(affineIni); 
                // since the model has Ns=0 it doesn't reflect light, so we change it to 1
                PhongMaterial material = (PhongMaterial) cubiePart.getMaterial();
                material.setSpecularPower(1);
                cubiePart.setMaterial(material);
                // finally, add the name of the part and the cubie part to the hashMap:
                mapMeshes.put(s,cubiePart); 
            });
        } catch (IOException e) {
            System.out.println("Error loading model "+e.toString());
        }
    }
    public Map<String, MeshView> getMapMeshes() {
        return mapMeshes;
    }
}

Since the model is oriented with white to the right (X axis) and red in the front (Z axis) (see picture above), two rotations are required: first rotate -90 degrees towards X axis, to put blue in the front, and then rotate 90 degrees arount Z axis to put white on top.

Mathematically, the second rotation matrix in Z must by multiplied on the left to the first matrix in X. But according to this link if we use add or append matrix rotations are operated on the right, and this will be wrong:

cubiePart.getTransforms().addAll(new Rotate(-90, Rotate.X_AXIS),
                                 new Rotate(90, Rotate.Z_AXIS));

as it will perform a first rotation in Z and a second one in X, putting red on top and yellow on front. Also this is wrong too:

cubiePart.getTransforms().addAll(new Rotate(90, Rotate.Z_AXIS),
                                  new Rotate(-90, Rotate.X_AXIS));

Though it does the right rotations, then it will require for further rotations of any cubie to be rotated from its original position, which is quite more complicated than rotating always from the last state.
           
So prepend is the right way to proceed here, and we just need to prepend the last rotation matrix to the Affine matrix of the cubie with all the previous rotations stored there.


Handling the model

After importing the obj file, we can figure out which is the number of each cubie, and once the cube it's well positioned (white face top, blue face front), the scheme we're going to use is a List<Integer> with 27 items:
  • first 9 indexes are the 9 cubies in the (F)Front face, from top left (R/W/B) to down right (Y/O/B).
  • second 9 indexes are from the (S)Standing face, from top left (R/W) to down right (Y/O).
  • last 9 indexes are from (B)Back face, from top left (G/R/W) to down right (G/Y/O).
But for performing rotations of these cubies, the best way is the internal use of a 3D array of integers:

    private final int[][][] cube={{{50,51,52},{49,54,53},{59,48,46}},
                                  {{58,55,60},{57,62,61},{47,56,63}},
                                  {{67,64,69},{66,71,70},{68,65,72}}};

where 50 is the number of the R/W/B cubie and 72 is the number for the G/Y/O.

The Rotations class will take care of any face rotation.

    // This is the method to perform any rotation on the 3D array just by swapping indexes
    // first index refers to faces F-S-B
    // second index refers to faces U-E-D
    // third index refers to faces L-M-R
    public void turn(String rot){ 
            int t = 0;
            for(int y = 2; y >= 0; --y){
                for(int x = 0; x < 3; x++){
                    switch(rot){
                        case "L":  tempCube[x][t][0] = cube[y][x][0]; break;
                        case "Li": tempCube[t][x][0] = cube[x][y][0]; break;
                        case "M":  tempCube[x][t][1] = cube[y][x][1]; break;
                        case "Mi": tempCube[t][x][1] = cube[x][y][1]; break;
                        case "R":  tempCube[t][x][2] = cube[x][y][2]; break;
                        case "Ri": tempCube[x][t][2] = cube[y][x][2]; break;
                        case "U":  tempCube[t][0][x] = cube[x][0][y]; break;
                        case "Ui": tempCube[x][0][t] = cube[y][0][x]; break;
                        case "E":  tempCube[x][1][t] = cube[y][1][x]; break;
                        case "Ei": tempCube[t][1][x] = cube[x][1][y]; break;
                        case "D":  tempCube[x][2][t] = cube[y][2][x]; break;
                        case "Di": tempCube[t][2][x] = cube[x][2][y]; break;
                        case "F":  tempCube[0][x][t] = cube[0][y][x]; break;
                        case "Fi": tempCube[0][t][x] = cube[0][x][y]; break;
                        case "S":  tempCube[1][x][t] = cube[1][y][x]; break;
                        case "Si": tempCube[1][t][x] = cube[1][x][y]; break;
                        case "B":  tempCube[2][t][x] = cube[2][x][y]; break;
                        case "Bi": tempCube[2][x][t] = cube[2][y][x]; break;
                    }
                }
                t++;
            }
        
        save();
    }

Similar rotations can be performed to the whole cube (X, Y or Z).

The content model

Once we have our model, we need a scene to display it. For that we'll use a SubScene object as content container, wrapped in a ContentModel class, where camera, lights and orientation axis are added, which is based in the ContentModel class from 3DViewer application:
 
public class ContentModel {
    public ContentModel(double paneW, double paneH, double dimModel) {
        this.paneW=paneW;
        this.paneH=paneH;
        this.dimModel=dimModel;
        buildCamera();
        buildSubScene();        
        buildAxes();
        addLights();        
    }

    private void buildCamera() {
        camera.setNearClip(1.0);
        camera.setFarClip(10000.0);
        camera.setFieldOfView(2d*dimModel/3d);
        camera.getTransforms().addAll(yUpRotate,cameraPosition,
                                      cameraLookXRotate,cameraLookZRotate);
        cameraXform.getChildren().add(cameraXform2);
        cameraXform2.getChildren().add(camera);
        cameraPosition.setZ(-2d*dimModel);
        root3D.getChildren().add(cameraXform);
        
        // Rotate camera to show isometric view X right, Y top, Z 120º left-down from each
        cameraXform.setRx(-30.0);
        cameraXform.setRy(30);

    }

    private void buildSubScene() {
        root3D.getChildren().add(autoScalingGroup);
        
        subScene = new SubScene(root3D,paneW,paneH,true,javafx.scene.SceneAntialiasing.BALANCED);
        subScene.setCamera(camera);
        subScene.setFill(Color.CADETBLUE);
        setListeners(true);
    }

    private void buildAxes() {
        double length = 2d*dimModel;
        double width = dimModel/100d;
        double radius = 2d*dimModel/100d;
        final PhongMaterial redMaterial = new PhongMaterial();
        redMaterial.setDiffuseColor(Color.DARKRED);
        redMaterial.setSpecularColor(Color.RED);
        final PhongMaterial greenMaterial = new PhongMaterial();
        greenMaterial.setDiffuseColor(Color.DARKGREEN);
        greenMaterial.setSpecularColor(Color.GREEN);
        final PhongMaterial blueMaterial = new PhongMaterial();
        blueMaterial.setDiffuseColor(Color.DARKBLUE);
        blueMaterial.setSpecularColor(Color.BLUE);
        
        Sphere xSphere = new Sphere(radius);
        Sphere ySphere = new Sphere(radius);
        Sphere zSphere = new Sphere(radius);
        xSphere.setMaterial(redMaterial);
        ySphere.setMaterial(greenMaterial);
        zSphere.setMaterial(blueMaterial);
        
        xSphere.setTranslateX(dimModel);
        ySphere.setTranslateY(dimModel);
        zSphere.setTranslateZ(dimModel);
        
        Box xAxis = new Box(length, width, width);
        Box yAxis = new Box(width, length, width);
        Box zAxis = new Box(width, width, length);
        xAxis.setMaterial(redMaterial);
        yAxis.setMaterial(greenMaterial);
        zAxis.setMaterial(blueMaterial);
        
        autoScalingGroup.getChildren().addAll(xAxis, yAxis, zAxis);
        autoScalingGroup.getChildren().addAll(xSphere, ySphere, zSphere);
    }
    
    private void addLights(){
        root3D.getChildren().add(ambientLight);
        root3D.getChildren().add(light1);
        light1.setTranslateX(dimModel*0.6);
        light1.setTranslateY(dimModel*0.6);
        light1.setTranslateZ(dimModel*0.6);
    }
}

For the camera, a Xform class from 3DViewer is used to change easily its rotation values. This also allows the initial rotation of the camera to show an isometric view:

    cameraXform.setRx(-30.0);
    cameraXform.setRy(30);

Other valid ways to perform these rotations could be based on obtaining the vector and angle of rotation to combine two rotations, which involve calculate the rotation matrix first and then the vector and angle (as I explained here):

    camera.setRotationAxis(new Point3D(-0.694747,0.694747,0.186157));
    camera.setRotate(42.1812);

Or prepending the two rotations to all the previous transformations, by appending all of them in a single Affine matrix before prepending these two rotations:

    Affine affineCamIni=new Affine();
    camera.getTransforms().stream().forEach(affineCamIni::append);
    affineCamIni.prepend(new Rotate(-30, Rotate.X_AXIS));
    affineCamIni.prepend(new Rotate(30, Rotate.Y_AXIS));
    camera.getTransforms().setAll(affineCamIni);

Then we add the listeners to the subscene, so the camera can be easily rotated.

    private void setListeners(boolean addListeners){
        if(addListeners){
            subScene.addEventHandler(MouseEvent.ANY, mouseEventHandler);
        } else {
            subScene.removeEventHandler(MouseEvent.ANY, mouseEventHandler);
        }
    }

    private final EventHandler<MouseEvent> mouseEventHandler = event -> {
        double xFlip = -1.0, yFlip=1.0; // y Up
        if (event.getEventType() == MouseEvent.MOUSE_PRESSED) {
            mousePosX = event.getSceneX();
            mousePosY = event.getSceneY();
            mouseOldX = event.getSceneX();
            mouseOldY = event.getSceneY();

        } else if (event.getEventType() == MouseEvent.MOUSE_DRAGGED) {
            double modifier = event.isControlDown()?0.1:event.isShiftDown()?3.0:1.0;

            mouseOldX = mousePosX;
            mouseOldY = mousePosY;
            mousePosX = event.getSceneX();
            mousePosY = event.getSceneY();
            mouseDeltaX = (mousePosX - mouseOldX);
            mouseDeltaY = (mousePosY - mouseOldY);

            if(event.isMiddleButtonDown() || (event.isPrimaryButtonDown() && event.isSecondaryButtonDown())) {
                cameraXform2.setTx(cameraXform2.t.getX() + xFlip*mouseDeltaX*modifierFactor*modifier*0.3); 
                cameraXform2.setTy(cameraXform2.t.getY() + yFlip*mouseDeltaY*modifierFactor*modifier*0.3);
            }
            else if(event.isPrimaryButtonDown()) {
                cameraXform.setRy(cameraXform.ry.getAngle() - yFlip*mouseDeltaX*modifierFactor*modifier*2.0);
                cameraXform.setRx(cameraXform.rx.getAngle() + xFlip*mouseDeltaY*modifierFactor*modifier*2.0);
            }
            else if(event.isSecondaryButtonDown()) {
                double z = cameraPosition.getZ();
                double newZ = z - xFlip*(mouseDeltaX+mouseDeltaY)*modifierFactor*modifier;
                cameraPosition.setZ(newZ);
            }
        }
    };

Handling the model

Now we can put all together and create the Rubik class, where the 3D model is imported, all the meshviews are created and grouped in cube, which is added to the content subscene. At the same time, rot is instantiated with the original position of the cubies.

public class Rubik {
    public Rubik(){
        // Import Rubik's Cube model and arrows
        Model3D model=new Model3D();
        model.importObj();
        mapMeshes=model.getMapMeshes();
        cube.getChildren().setAll(mapMeshes.values());
        dimCube=cube.getBoundsInParent().getWidth();
        
        // Create content subscene, add cube, set camera and lights
        content = new ContentModel(800,600,dimCube); 
        content.setContent(cube);
                
        // Initialize 3D array of indexes and a copy of original/solved position
        rot=new Rotations();
        order=rot.getCube();

        // save original position
        mapMeshes.forEach((k,v)->mapTransformsOriginal.put(k, v.getTransforms().get(0)));
        orderOriginal=order.stream().collect(Collectors.toList());
        
        // Listener to perform an animated face rotation
        rotMap=(ov,angOld,angNew)->{ 
            mapMeshes.forEach((k,v)->{
                layer.stream().filter(l->k.contains(l.toString()))
                    .findFirst().ifPresent(l->{
                        Affine a=new Affine(v.getTransforms().get(0));
                        a.prepend(new Rotate(angNew.doubleValue()-angOld.doubleValue(),axis));
                        v.getTransforms().setAll(a);
                    });
            });
        };
    }
}

Finally we create a listener for rotating layers of cubies in a Timeline animation. As the rotations are prepended to the actual affine matrix of the cubies, to perform a smooth animation we'll change the angle between 0 and 90º, and listen how the timeline internally interpolate it, making small rotations between angNew and angOld angles.

So the method to perform the rotation could be like this:

    public void rotateFace(final String btRot){
        if(onRotation.get()){
            return;
        }
        onRotation.set(true);
        
        // rotate cube indexes
        rot.turn(btRot);
        // get new indexes in terms of blocks numbers from original order
        reorder=rot.getCube();
        // select cubies to rotate: those in reorder different from order.
        AtomicInteger index = new AtomicInteger();
        layer=order.stream()
                   .filter(o->!Objects.equals(o, reorder.get(index.getAndIncrement())))
                   .collect(Collectors.toList());
        // add central cubie
        layer.add(0,reorder.get(Utils.getCenter(btRot)));
        // set rotation axis            
        axis=Utils.getAxis(btRot); 
        
        // define rotation
        double angEnd=90d*(btRot.endsWith("i")?1d:-1d);
        
        rotation.set(0d);
        // add listener to rotation changes
        rotation.addListener(rotMap);

        // create animation
        Timeline timeline=new Timeline();
        timeline.getKeyFrames().add(
            new KeyFrame(Duration.millis(600), e->{
                    // remove listener
                    rotation.removeListener(rotMap);
                    onRotation.set(false); 
                },  new KeyValue(rotation,angEnd)));
        timeline.playFromStart();

        // update order with last list
        order=reorder.stream().collect(Collectors.toList());
    }

RubikFX, Lite Version

Later on we'll add more features, but for now let's create a JavaFX application, with a BorderPane, add content to the center of the pane, and a few toolbars with buttons to perform rotations.

public class TestRubikFX extends Application {
    
    private final BorderPane pane=new BorderPane();
    private Rubik rubik;
    
    @Override
    public void start(Stage stage) {
        rubik=new Rubik();
        // create toolbars
        ToolBar tbTop=new ToolBar(new Button("U"),new Button("Ui"),new Button("F"),
                                  new Button("Fi"),new Separator(),new Button("Y"),
                                  new Button("Yi"),new Button("Z"),new Button("Zi"));
        pane.setTop(tbTop);
        ToolBar tbBottom=new ToolBar(new Button("B"),new Button("Bi"),new Button("D"),
                                     new Button("Di"),new Button("E"),new Button("Ei"));
        pane.setBottom(tbBottom);
        ToolBar tbRight=new ToolBar(new Button("R"),new Button("Ri"),new Separator(),
                                    new Button("X"),new Button("Xi"));
        tbRight.setOrientation(Orientation.VERTICAL);
        pane.setRight(tbRight);
        ToolBar tbLeft=new ToolBar(new Button("L"),new Button("Li"),new Button("M"),
                                   new Button("Mi"),new Button("S"),new Button("Si"));
        tbLeft.setOrientation(Orientation.VERTICAL);
        pane.setLeft(tbLeft);
        
        pane.setCenter(rubik.getSubScene());
        
        pane.getChildren().stream()
            .filter(n->(n instanceof ToolBar))
            .forEach(tb->{
                ((ToolBar)tb).getItems().stream()
                    .filter(n->(n instanceof Button))
                    .forEach(n->((Button)n).setOnAction(e->rubik.rotateFace(((Button)n).getText())));
            });
        rubik.isOnRotation().addListener((ov,b,b1)->{
            pane.getChildren().stream()
            .filter(n->(n instanceof ToolBar))
            .forEach(tb->tb.setDisable(b1));
        });
        final Scene scene = new Scene(pane, 880, 680, true);
        scene.setFill(Color.ALICEBLUE);
        stage.setTitle("Rubik's Cube - JavaFX3D");
        stage.setScene(scene);
        stage.show();
    } 
}

Now this is what we have already accomplished:


If you're interested in having a deeper look at the application, you can find the source code in my GitHub repository. Note you'll need to add the 3DViewer.jar. 

How does it work?

Take, for instance, a initial "F" rotation. We apply it to rot:


        // rotate cube indexes
        rot.turn(btRot);
        // get new indexes in terms of blocks numbers from original order
        reorder=rot.getCube();

Using rot.printCube() we can see the numbers of cubies for the solved cube (order) and for the new one, with the frontal layer rotated clockwise (reorder):

order:    50 51 52 49 54 53 59 48 46 || 58 55 60 57 62 61 47 56 63 || 67 64 69 66 71 70 68 65 72
reorder:  59 49 50 48 54 51 46 53 52 || 58 55 60 57 62 61 47 56 63 || 67 64 69 66 71 70 68 65 72


By comparing both lists and getting the different items, we know which cubies must be rotated, though we have to add the number of the central cubie (54), as it is the same in both lists, but it should be rotated too. So we create the list layer with these nine cubies:

    // select cubies to rotate: those in reorder different from order.
        AtomicInteger index = new AtomicInteger();
        layer=order.stream()
                   .filter(o->!Objects.equals(o, reorder.get(index.getAndIncrement())))
                   .collect(Collectors.toList());
        // add central cubie
        layer.add(0,reorder.get(Utils.getCenter(btRot)));
        // set rotation axis            
        axis=Utils.getAxis(btRot); 

Utils is a class that manage the values for each type of rotation. For this case:

   public static Point3D getAxis(String face){
        Point3D p=new Point3D(0,0,0);
        switch(face.substring(0,1)){
            case "F":  
            case "S":  p=new Point3D(0,0,1); 
                       break;
        }
        return p;
    }
    
    public static int getCenter(String face){
        int c=0;
        switch(face.substring(0,1)){
            case "F":  c=4;  break;
        }
        return c;
    }

Once we've got the cubies and the axis of rotation, now it's worth noticing how the rotation listener works. With the timeline, the angle goes from 0 to 90º with an EASE_BOTH interpolation (by default), so the angle increments are smaller at the beginning, bigger in the middle and smaller again at the end. This could be a possible list of increments: 0.125º-3º-4.6º-2.2º-2.48º-...-2.43º-4.78º-2.4º-2.4º-0.55º.

For every value in angNew, the listener rotMap applies a small rotation to a layer of cubies. For that we look in our HashMap which meshviews belongs to these cubies, and prepend a new rotation to their previous affine matrix:

   // Listener to perform an animated face rotation
        rotMap=(ov,angOld,angNew)->{ 
            mapMeshes.forEach((k,v)->{
                layer.stream().filter(l->k.contains(l.toString()))
                    .findFirst().ifPresent(l->{
                        Affine a=new Affine(v.getTransforms().get(0));
                        a.prepend(new Rotate(angNew.doubleValue()-angOld.doubleValue(),axis));
                        v.getTransforms().setAll(a);
                    });
            });
        };

So in 600 ms we apply around 30 to 40 small rotations to a bunch of around 40 meshviews. 

Finally, after the rotation is done, we just need to update order with the last list of cubies, so we can start all over again with a new rotation.


2. The Rubik's Cube - Full Version

 Adding more features

Now that we've got a working but pretty basic Rubik's cube JavaFX application, it's time for adding a few extra features, like graphic arrows and preview rotations to show the direction of rotation before they're performed.

Scramble and Sequences

Let's start by adding a scramble routine, to scramble the cubies before start solving the cube. To do that we generate a sequence of 25 random moves from a list of valid rotations.

   private static final List<String> movements = 
    Arrays.asList("F", "Fi", "F2", "R", "Ri", "R2", 
                  "B", "Bi", "B2", "L", "Li", "L2",
                  "U", "Ui", "U2", "D", "Di", "D2");

    private String last="V", get="V";
    public void doScramble(){
        StringBuilder sb=new StringBuilder();
        IntStream.range(0, 25).boxed().forEach(i->{
            while(last.substring(0, 1).equals(get.substring(0, 1))){
                // avoid repeating the same/opposite rotations
                get=movements.get((int)(Math.floor(Math.random()*movements.size())));
            }
            last=get;
            if(get.contains("2")){
                get=get.substring(0,1);
                sb.append(get).append(" ");
            }
            sb.append(get).append(" ");
        });
        doSequence(sb.toString().trim());
    }

Then we have to perform this sequence, by rotating each movement.  First we extract the rotations from the string, converting other notations (like lower letters or ' instead of 'i' for counter clockwise rotations) to the one used.

A listener is added to onRotation, so only when the last rotation finishes, a new rotation starts. By adding a second listener to the index property, when the end of the list plus one is reached, this listener is stopped, allowing for the last rotation to finish properly, and saving the rotations for a further replay option.

   public void doSequence(String list){
        onScrambling.set(true);
        List<String> asList = Arrays.asList(list.replaceAll("’", "i").replaceAll("'", "i").split(" "));
        
        sequence=new ArrayList<>();
        asList.stream().forEach(s->{
            if(s.contains("2")){
                sequence.add(s.substring(0, 1));
                sequence.add(s.substring(0, 1));            
            } else if(s.length()==1 && s.matches("[a-z]")){
                sequence.add(s.toUpperCase().concat("i"));
            } else {
                sequence.add(s);
            }
        });
        System.out.println("seq: "+sequence);
        
        IntegerProperty index=new SimpleIntegerProperty(1);
        ChangeListener<boolean> lis=(ov,b,b1)->{
            if(!b1){
                if(index.get()<sequence.size()){
                    rotateFace(sequence.get(index.get()));
                } else {
                    // save transforms
                    mapMeshes.forEach((k,v)->mapTransformsScramble.put(k, v.getTransforms().get(0)));
                    orderScramble=reorder.stream().collect(Collectors.toList());
                } 
                index.set(index.get()+1);
            }
        };
        index.addListener((ov,v,v1)->{
            if(v1.intValue()==sequence.size()+1){
                onScrambling.set(false);
                onRotation.removeListener(lis);
                count.set(-1);
            }
        });
        onRotation.addListener(lis);
        rotateFace(sequence.get(0));
    }

Note that we use a Dialog from ControlsFX to prevent losing previous moves.


   Button bSc=new Button("Scramble");
        bSc.setOnAction(e->{
            if(moves.getNumMoves()>0){
                Action response = Dialogs.create()
                .owner(stage)
                .title("Warning Dialog")
                .masthead("Scramble Cube")
                .message( "You will lose all your previous movements. Do you want to continue?")
                .showConfirm();
                if(response==Dialog.Actions.YES){
                    rubik.doReset();
                    doScramble();
                }
            } else {
                doScramble();
            }
        });

If you want to load a sequence, like any of these, another Dialog with input allowed is used.

   Button bSeq=new Button("Sequence");
        bSeq.setOnAction(e->{
            String response;
            if(moves.getNumMoves()>0){
                response = Dialogs.create()
                .owner(stage)
                .title("Warning Dialog")
                .masthead("Loading a Sequence").lightweight()
                .message("Add a valid sequence of movements:\n(previous movements will be discarded)")
                .showTextInput(moves.getSequence());
            } else {
                response = Dialogs.create()
                .owner(stage)
                .title("Information Dialog")
                .masthead("Loading a Sequence").lightweight()
                .message( "Add a valid sequence of movements")
                .showTextInput();
            }
            if(response!=null && !response.isEmpty()){
                rubik.doReset();
                rubik.doSequence(response.trim());
            }
        });

The results of scrambling a cube or adding a sequence of rotations can be seen in this video.



Timer and moves counter

Let's add now a timer using the new Date and Time API for Java 8. You may have noticed the timer in the bottom toolbar in the previous video.

For that, we use the following code in RubikFX class:


    private LocalTime time=LocalTime.now();
    private Timeline timer;
    private final StringProperty clock = new SimpleStringProperty("00:00:00");
    private final DateTimeFormatter fmt = DateTimeFormatter.ofPattern("HH:mm:ss").withZone(ZoneId.systemDefault());
    
    @Override
    public void start(Stage stage) {
        ...
        Label lTime=new Label();
        lTime.textProperty().bind(clock);
        tbBottom.getItems().addAll(new Separator(),lTime);

        timer=new Timeline(new KeyFrame(Duration.ZERO, e->{
            clock.set(LocalTime.now().minusNanos(time.toNanoOfDay()).format(fmt));
        }),new KeyFrame(Duration.seconds(1)));
        timer.setCycleCount(Animation.INDEFINITE);

        rubik.isSolved().addListener((ov,b,b1)->{
            if(b1){
                timer.stop();
            }
        });
        
        time=LocalTime.now();
        timer.playFromStart();
    }

For the counter, we'll add two classes. Move is a simple POJO class, with a string for the name of the rotation and a long for the timestamp of the movement. Moves class will contain a list of moves.


public class Moves {
    private final List<Move> moves=new ArrayList<>();
    
    public Moves(){
        moves.clear();
    }
    
    public void addMove(Move m){ moves.add(m); }
    public List<Move> getMoves() { return moves; }
    public Move getMove(int index){
        if(index>-1 && index<moves.size()){
            return moves.get(index);
        }
        return null;
    }
    public String getSequence(){
        StringBuilder sb=new StringBuilder("");
        moves.forEach(m->sb.append(m.getFace()).append(" "));
        return sb.toString().trim();
    }
}

For adding the number of rotations, we use the following code in RubikFX class:


    private Moves moves=new Moves();
    
    @Override
    public void start(Stage stage) {
        ...
        rubik.getLastRotation().addListener((ov,v,v1)->{
            if(!v1.isEmpty()){
                moves.addMove(new Move(v1, LocalTime.now().minusNanos(time.toNanoOfDay()).toNanoOfDay()));
            }
        });

        Label lMov=new Label();
        rubik.getCount().addListener((ov,v,v1)->{
            lMov.setText("Movements: "+(v1.intValue()+1));
        });
        tbBottom.getItems().addAll(new Separator(),lMov);
    }

Replay

We can also replay the list of movements the user has done stored in moves. For that we need to restore first the state of the cube right after the scramble, and performe one by one all the rotations from the list.


    public void doReplay(List<Move> moves){
        if(moves.isEmpty()){
            return;
        }
        content.resetCam();
        //restore scramble
        if(mapTransformsScramble.size()>0){
            mapMeshes.forEach((k,v)->v.getTransforms().setAll(mapTransformsScramble.get(k)));
            order=orderScramble.stream().collect(Collectors.toList());
            rot.setCube(order);
            count.set(-1);
        } else {
            // restore original
            doReset();
        }

        onReplaying.set(true);        
        IntegerProperty index=new SimpleIntegerProperty(1);
        ChangeListener<boolean> lis=(ov,v,v1)->{
            if(!v1 && moves.size()>1){
                if(index.get()<moves.size()){
                    timestamp.set(moves.get(index.get()).getTimestamp());
                    rotateFace(moves.get(index.get()).getFace());
                }
                index.set(index.get()+1);
            }
        };
        index.addListener((ov,v,v1)->{
            if(v1.intValue()==moves.size()+1){
                onReplaying.set(false);
                onRotation.removeListener(lis);
                acuAngle=0;
            }
        });
        onRotation.addListener(lis);
        timestamp.set(moves.get(0).getTimestamp());
        rotateFace(moves.get(0).getFace());
    }

Rotation direction preview

Time for a new feature: 3D arrows will be shown in the rotating face or axis, to show the direction.

Actually, JavaFX 3D API doesn't supply any way of building 3D complex models. There's an impressive ongoing work by Michael Hoffer to provide a way by using Constructive Solid Geometry (CSG) here, kudos Michael!!

By using primitives and boolean operations with CSG you can build a model, and even export it with STL format.




You can use free or commercial 3D software for this task too. I designed these arrows with SketchUp Make and exported them to OBJ format so I could import them as we did with the cube using ObjImporter from 3DViewer. 



While the design is fast, it requires manual editting of the created obj file to convert long faces of more than 4 vertixes as they are not properly imported.



Other approach could be exporting the file to *.3ds and use the proper importer from August Lammersdorf.  


Edit: Michael Hoffer kindly added an option to export to OBJ format, so now it would be possible to import the arrow model generated with CSG in JavaFXScad in our scene. Thanks Michael!

Once we have the model, we have to add it, scale and rotate it, so we can show the arrow in the rotating face.




For a rotation like 'Ui':
 
    public void updateArrow(String face, boolean hover){
        boolean bFaceArrow=!(face.startsWith("X")||face.startsWith("Y")||face.startsWith("Z"));
        MeshView arrow=bFaceArrow?faceArrow:axisArrow;
        
        if(hover && onRotation.get()){
            return;
        }
        arrow.getTransforms().clear();    
        if(hover){
            double d0=arrow.getBoundsInParent().getHeight()/2d;
            Affine aff=Utils.getAffine(dimCube, d0, bFaceArrow, face);
            arrow.getTransforms().setAll(aff);
            arrow.setMaterial(Utils.getMaterial(face));
            if(previewFace.get().isEmpty()) {
                previewFace.set(face);
                onPreview.set(true);
                rotateFace(face,true,false);
            }
        } else if(previewFace.get().equals(face)){
            rotateFace(Utils.reverseRotation(face),true,true);
        } else if(previewFace.get().equals("V")){
            previewFace.set("");
            onPreview.set(false);
        }
    }

where the affine is calculated in the Utils class for the current face:

    public static Affine getAffine(double dimCube, double d0, boolean bFaceArrow, String face){
        Affine aff=new Affine(new Scale(3,3,3));
        aff.append(new Translate(0,-d0,0));
        switch(face){
            case "U":   
            case "Ui":  aff.prepend(new Rotate(face.equals("Ui")?180:0,Rotate.Z_AXIS));
                        aff.prepend(new Rotate(face.equals("Ui")?45:-45,Rotate.Y_AXIS));
                        aff.prepend(new Translate(0,dimCube/2d,0));
                        break;
        }
        return aff;
    }

To trigger the drawing of the arrow, we set a listener to the buttons on the toolbars based on the mouse hovering.

We can also add a small rotation (5º) as preview of the full rotation (90º) in the face selected, by calling rotateFace again, with bPreview=true at this point.



If the user clicks on the button, the rotation is completed (from 5º to 90º). Otherwise the rotation is cancelled (from 5º to 0º). In both cases, with a smooth animation.



Select rotation by picking

Finally, the rotation could be performed based on the mouse picking of a cubie face, with visual aid showing the arrow and performing a small rotation of 5º. If the mouse is dragged far enough the full rotation will be performed after it is released. If the mouse is released close to the origin, the rotation is cancelled.

For this feature, the critical part is being able to know which mesh we are selecting with the mouse click. And for that, the API provides MouseEvent.getPickResult().getIntersectedNode(), which returns one of the meshviews on the cube.

So the next step is find which is this meshview and what cubie does it belongs to. As all the meshes have a name, like 'Block46 (2)', looking at the number of block we identify the cubie.

Now we need to find which of the faces we have selected. For that we use the triangles coordinates of the mesh, as for the faces they define a plane, so with the cross product we know the normal direction of this plane. Note we must update the operations with the actual set of transformations applied.

    private static Point3D getMeshNormal(MeshView mesh){
        TriangleMesh tm=(TriangleMesh)mesh.getMesh();
        float[] fPoints=new float[tm.getPoints().size()];
        tm.getPoints().toArray(fPoints);
        Point3D BA=new Point3D(fPoints[3]-fPoints[0],fPoints[4]-fPoints[1],fPoints[5]-fPoints[2]);
        Point3D CA=new Point3D(fPoints[6]-fPoints[0],fPoints[7]-fPoints[1],fPoints[8]-fPoints[2]);
        Point3D normal=BA.crossProduct(CA);
        Affine a=new Affine(mesh.getTransforms().get(0));
        return a.transform(normal.normalize());
    }

    public static String getPickedRotation(int cubie, MeshView mesh){
        Point3D normal=getMeshNormal(mesh);
        String rots=""; // Rx-Ry 
        switch(cubie){
            case 0: rots=(normal.getZ()>0.99)?"Ui-Li":
                            ((normal.getX()<-0.99)?"Ui-F":((normal.getY()>0.99)?"Ui-Li":""));
                    break;
        }
        return rots;
    }

Once we have the normal, we can provide the user with two possible rotations (and their possible two directions). To select which one to perform, we'll look how the user moves the mouse while it's being dragged. Note the mouse coordinates are 2D.


    public static String getRightRotation(Point3D p, String selFaces){
        double radius=p.magnitude();
        double angle=Math.atan2(p.getY(),p.getX());
        String face="";
        if(radius>=radMinimum && selFaces.contains("-") && selFaces.split("-").length==2){
            String[] faces=selFaces.split("-");
            // select rotation if p.getX>p.getY
            if(-Math.PI/4d<=angle && angle<Math.PI/4d){ // X
                face=faces[0];
            } else if(Math.PI/4d<=angle && angle<3d*Math.PI/4d){ // Y
                face=faces[1];
            } else if((3d*Math.PI/4d<=angle && angle<=Math.PI) || 
                      (-Math.PI<=angle && angle<-3d*Math.PI/4d)){ // -X
                face=reverseRotation(faces[0]);
            } else { //-Y
                face=reverseRotation(faces[1]);
            }
            System.out.println("face: "+face);
        } else if(!face.isEmpty() && radius<radMinimum){ // reset previous face
            face="";
        }
        return face;
    }

Now that we have the layer to rotate, we make a small rotation as a preview of rotation if the mouse is dragged far from the initial click point, with a minimum distance. Then if the user releases the mouse and the distance from the initial point is greater than a distance radClick, the rotation is completed. But if the distance is lower or the mouse is dragged under the distance radMinimum, the rotation is cancelled.

The next listing shows an EventHandler<MouseEvent> implemented to provide this behaviour. Note that we have to stop the camera rotations while we are picking a face and rotating a layer.


    public EventHandler<MouseEvent> eventHandler=(MouseEvent event)->{
            if (event.getEventType() == MouseEvent.MOUSE_PRESSED ||
                event.getEventType() == MouseEvent.MOUSE_DRAGGED || 
                event.getEventType() == MouseEvent.MOUSE_RELEASED) {
                
                mouseNewX = event.getSceneX();
                mouseNewY = -event.getSceneY();
                
                if (event.getEventType() == MouseEvent.MOUSE_PRESSED) {
                    Node picked = event.getPickResult().getIntersectedNode();
                    if(null != picked && picked instanceof MeshView) {
                        mouse.set(MOUSE_PRESSED);
                        cursor.set(Cursor.CLOSED_HAND);
                        stopEventHandling();
                        stopEvents=true;
                        pickedMesh=(MeshView)picked;
                        String block=pickedMesh.getId().substring(5,7);
                        int indexOf = order.indexOf(new Integer(block));
                        selFaces=Utils.getPickedRotation(indexOf, pickedMesh);
                        mouseIniX=mouseNewX;
                        mouseIniY=mouseNewY;
                        myFace=""; 
                        myFaceOld="";
                    }
                } else if (event.getEventType() == MouseEvent.MOUSE_DRAGGED) {
                    if(stopEvents && !selFaces.isEmpty()){
                        mouse.set(MOUSE_DRAGGED);
                        Point3D p=new Point3D(mouseNewX-mouseIniX,mouseNewY-mouseIniY,0);
                        radius=p.magnitude();

                        if(myFaceOld.isEmpty()){
                            myFace=Utils.getRightRotation(p,selFaces);
                            if(!myFace.isEmpty() && !onRotation.get()){
                                updateArrow(myFace, true);
                                myFaceOld=myFace;
                            } 
                            if(myFace.isEmpty()){
                                myFaceOld="";
                            }
                        }
                        // to cancel preselection, just go back to initial click point
                        if(!myFaceOld.isEmpty() && radius<Utils.radMinimum){
                            myFaceOld="";
                            updateArrow(myFace, false);
                            myFace="";
                        }
                    }
                } else if (stopEvents && event.getEventType() == MouseEvent.MOUSE_RELEASED) {
                    mouse.set(MOUSE_RELEASED);
                    if(!onRotation.get() && !myFace.isEmpty() && !myFaceOld.isEmpty()){
                        if(Utils.radClick<radius){
                            // if hand is moved far away do full rotation
                            rotateFace(myFace);
                        } else { 
                            // else preview cancellation
                            updateArrow(myFace, false);
                        }
                    }
                    myFace=""; myFaceOld="";
                    stopEvents=false;
                    resumeEventHandling();                        
                    cursor.set(Cursor.DEFAULT);
                }
            }
        };

Finally, we add this EventHandler to the scene:


scene.addEventHandler(MouseEvent.ANY, rubik.eventHandler);

This video shows how this event handling works.


 


Check if cube is solved

Finally, let's add a check routine. We know the initial solved order of cubies, but we need to take into account any of the 24 possible orientations of the faces, which can be acchieved with up to two rotations.

    private static final List<String> orientations=Arrays.asList("V-V","V-Y","V-Yi","V-Y2",
                                                    "X-V","X-Z","X-Zi","X-Z2",
                                                    "Xi-V","Xi-Z","Xi-Zi",
                                                    "X2-V","X2-Z","X2-Zi",
                                                    "X-Y","X-Yi","X-Y2",
                                                    "Xi-Y","Xi-Yi","X2-Y","X2-Yi",
                                                    "Z-V","Zi-V","Z2-V");
    
    public static boolean checkOrientation(String r, List<Integer> order){
        Rotations rot=new Rotations();
        for(String s:r.split("-")){
            if(s.contains("2")){
                rot.turn(s.substring(0,1));
                rot.turn(s.substring(0,1));
            } else {
                rot.turn(s);
            }
        }
        return order.equals(rot.getCube());
    }

So after any movement we have to check if the actual order matches any of these 24 solutions. For that we can use parallelStream() with a filter in which we rotate a new cube to one of the possible orientations and check if that matches the actual one:


    public static boolean checkSolution(List<Integer> order) {
        return Utils.getOrientations().parallelStream()
                .filter(r->Utils.checkOrientation(r,order)).findAny().isPresent();
    }
Conclusions

All along this post, we've been discussing the new JavaFX 3D API. Powerfull enough, but with lack of some usual tools in the 3D modelling world. Maybe they will come soon...

We've used the new lambdas and Stream API. I hope by now you've got a clear view of what you can do with them. For sure, they will definetely change the way we write code.

The Rubik's cube application has proven to be a nice way of testing these new capabilities, while enjoying playing, humm, I mean, developing the code.

This final video shows most of what we've accomplished in this post. As I said, it's for begginers like me with the Rubik's cube...


In my repo you can find all the code for this full version. Fill free to fork it and play with it. There're tons of improvements to make, so any pull request will be welcome!



Edit: And if you just want to try it before having a look at the code, here you can download a single executable jar. Download it and run it with Java 8 installed.

As always, thanks for reading me! Please, try it for yourself and share any comment you may have.