Personal R&D · Unity · Python · Keras
Predicting light angles with synthetic data
Can a small neural network look at a rendered image and tell you where the light is coming from? I generated a training set from Unity renders and trained a convolutional network (CNN) to predict the angle of a directional light. The tool aims to help with lighting analysis and with matching virtual lighting in AR/VR.
1 · Generating the data in Unity
A script rotates a directional light around the Y axis and captures a frame at each step. I limited the range to −150° to 150°, where the camera can actually see reflections and shading change, and sampled it every 2.5°.
directionalLight.transform.rotation = Quaternion.Euler(0, angleY, 0);
CaptureAndSaveImage(angleY);2 · The network
The network takes 128×128 images through convolution and max-pooling layers into dense layers, and outputs a single angle. Dropout and a learning rate schedule keep it from overfitting.
model = Sequential([
Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(128, 128, 3)),
MaxPooling2D(pool_size=(2, 2)),
Conv2D(64, kernel_size=(3, 3), activation='relu'),
MaxPooling2D(pool_size=(2, 2)),
Flatten(),
Dense(128, activation='relu'),
Dropout(0.5),
Dense(1)
])I normalized the images to the range 0 to 1 and split them into training, validation, and test sets.
3 · What went wrong, and the fixes
- Useless samples. Early datasets included light angles that produced no visible change from the camera's position, which only added noise. Restricting the range to −150° to 150° fixed both training speed and accuracy.
- Flat predictions. At first the model predicted nearly the same value for every input. Finer sampling, exponential learning rate decay, and early stopping got it learning the actual relationship.
4 · Results
The model reached a validation mean absolute error of about 8.7 on test images it had never seen. Plots of predicted against true angles show the outputs following the real values.
Where it could go
- Lighting estimation for AR/VR: matching virtual lighting to a real scene so inserted objects sit naturally in it.
- Automated lighting analysis: checking in-game lighting setups and suggesting adjustments.
- Procedural lighting: adapting light to generated geometry and player movement.
- Next steps: more complex scenes, a wider range of materials, and domain adaptation to real photographs.