Inspiration
The search for new and improved designs is a never-ending process, whether it's an automobile, architecture or any machine around us. Engineering Design depends on computers, i.e the Computer Aided Design (CAD) framework. An Initial layout is essential for the success of CAD, usually, the initial layout is chosen from the designer’s past experience or rules from literature. However, this leads to bias from experience to creep into the design process thereby limiting innovative design solutions or unexpected designs to be overlooked.
What it does
This project aims to implement recent advances in AI to arrive at novel solutions to engineering design problems. Our solution optimizes the material distribution of an electromagnetic device to improve certain performance criteria while subject to some design constraints.
How we built it
We first gamified the simulator to act as an oracle which would provide rewards to our RL Agent.
Then we spent time understanding the gradient flow from critic to the actor in DDPG and similar actor-critic algorithms and converted it into build-graph-first programming paradigm of tensorflow.
We also broke our initial monolithic code into 5 different files making it modular and easy to understand.
Challenges we ran into
We attempted to implement two reinforcement algorithms to solve our Material Distribution problem: Modified Deep Deterministic Policy Gradient (DDPG) and Actor-Critic.
Although we obtained encouraging results from the Modified DDPG experiments, the Actor-Critic algorithm had bugs which could not be resolved within the required time frame.
There was no RL friendly environment readily available for our project. We had to design an environment from scratch to test the RL algorithms.
We did not find any baselines to begin with since RL is being applied to this problem for the first time, to the best of our knowledge.
Searching and optimizing the hyper-parameters for training the neural networks.
Accomplishments that we're proud of
Formulating the Material Distribution problem as a game with rewards.
Designing the environment and implementing the Modified DDPG algorithm.
Achieving simulation results which are better than the baseline performance.
What we learned
Designing known RL algorithms to conform to the Material Design use-case.
Understood and implemented Modified DDPG and Actor-Critic methods.
Learning to work with the different tools used in Machine Learning: Python, Tensorflow and so on. Running GPU instances on Azure.
What's next for designer_RL
Fine-tuning the hyperparameters to improve the performance of the ActorCritic algorithm.
Experimentation with other RL algorithms (such as A3C).
Experimentation with various game formulations.
Expanding the environment to achieve a solution for a generalized 2D EM design problem.
Efficient solutions for 3D EM design optimization.
Log in or sign up for Devpost to join the conversation.