18th Apr 2018

Object interaction in ARCore for android

Mobile App

Written By, Sumeet Rukeja


After a long wait, google has finally released a stable version of ARCore sdk for android.
But it hasn’t disappointed us from functionality point of view. The brand new sdk is capable to detect vertical as well as horizontal surfaces unlike prior couple of developer versions of sdk which were just able to detect horizontal surfaces.

ARCore SDK 1.0


Latest sdk supports a longer range of devices including Asus and Huawei products. But roses often come with thorns!

Google has imposed a little restriction on memory usage of devices while running AR apps which forces us to use background threads for heavy operations like changing texture of objects at run time.
This is obvious step in order to support devices with lower memory.
In earlier versions of sdk, memory of devices was kind of vulnerable. So, as a conclusion, I’ll consider this positive.

Google has also released emulators to test apps. Isn’t it mind boggling?


I have kept an eye on ARCore since the day Google released its developer preview version. The sample app provided by Google uses OpenGL to read and load objects. But, it just shows us how to place objects.
As a developer, one cannot stay satisfied with such simple functionality!
As I’m a beginner in OpenGL for android, I searched a lot to add object interaction functionality in the app.

I’ll show you how to rotate, scale and move the objects around and an eye-catching functionality – changing objects at run time.


Initially, you will need the ARCore sample app provided by Google, an emulator which supports ARCore app or an ARCore supported devices and 3D models with their textures.
I’ll not dive into the deep ocean of ARCore and OpenGl, explaining the very basics. The only thing to keep in mind is, ARCore places points in real world and tracks them. The task of drawing and moving objects is solely based on graphics libraries, in this case: OpenGL.

Loading object into ARCore

Place models and their textures in assets folder.


Here, andy and shoes are our two models.
Now, declare two String variables in MainActivity.java to hold the values of object file and texture file. Initialize them as shown below.

In onSurfaceCreated( ) method, use these variables to create model.

Note: In onDraw( ) method, change the if-condition to restrict the user to place one object at a time.

Rotating object

I have used a helper class to detect pinch-to-rotate gesture. Based on requirement, any gesture can be used here.

We have to implement its OnRotationGestureListener( ) in our main class.

Declare a static public float variable to store the rotation value, in this case: GlobalClass.rotateF.

Now, in ObjectRenderer.java class, declare and initialize a matrix (4 x 4 array).

In draw( ) method, edit code as shown below.

Matrix.setRotateM(mRotationMatrix, 0, GlobalClass.rotateF, 0.0f, 1.0f, 0.0f);
Here, I have used the rotation factor to change the matrix of the object along y-axis.
4th, 5th and 6th arguments are for x, y and z axis respectively.


Use the final matrix as in the code below.

Moving object

This is actually a workaround to translate the object along the surface.
Redrawing the object in onScroll( ) method of surface view does this trick.

Here, I have used a counter to calculate number of touches on the surface. If there are less than 2 fingers on the surface, the object will get redrawn on the point. Hence, scrolling gesture will produce a translating effect on object.

Scaling object

To implement this functionality, we have used onDoubleTap( ) listener.

We need to store scale factor in a public static variable, in this case: GlobalClass.scaleFactor.

This factor is used in onDrawFrame( ) method of main activity. This method is executed continuously in fraction of a second.

updateModelMatrix( ) is a method of object renderer class which sets the scale of the model.

I have increased the factor’s value in the method.

Changing object at run time

Here comes the coolest functionality. We got a button, clicking on it will change the model. All we have to do is change the values of String variables which we declared for storing model name and texture.

We have to keep a track of whether the model is changed. isObjectReplaced is used for it.

In onDrawFrame( ) method, we have to add the below code. This will replace the object.

Tip: To read model and textures from sd card, replace code in createIOnGlThread( )

Here, at Yudiz, we are concentrating on advanced ARCore topics like managing multiple objects simultaneously, selecting models using touch gestures.


ARCore is Google’s answer to Apple’s ARKit. I personally think that it will dominate in AR field as it has great potential.

Written By,

Android Developer at Yudiz Solutions Pvt. Ltd