DeepRacer Reflection
And a big CONGRATULATIONS to all of the winners!
- First-place: Acuity Tech Intern Joycelyn Joo – 60 seconds
- Second-place: Senior Tech Consultant David Gornak – 76 seconds
- Third-place tie: Senior Manager Kristin Casad and Senior Tech Manager Jesse Adams – 78 seconds
- Honorable Mention: Jessica Alfaro – 79 seconds
Great race everyone! The top place winners will all receive a medal and Amazon Echo Dots, while the first-place winner will have her name engraved on the Acuity DeepRacer trophy. Jessica Alfaro was also chosen to attend AWS re:Invent in the fall; you can see her last VLOG entry below.
A special thank you to AWS for coming down and helping us run the event and providing sweet T-shirts and the Echo Dots for prizes.
For a look back at all of the excitement check out our highlight video:
We also wanted to share the perspective of one Acuity DeepRacer, Senior Technical Manager Ryan Poole, as he describes his journey and his lessons learned below:
When I was asked to submit a write-up for my journey, I was wondering what to include and where do I start?
Well…I need to start with a thank you to Jesse Adams for encouraging me to get involved in DeepRacer. Jesse said it would be fun and I would have to agree. However, he didn’t mention the challenging aspect of this journey as well. 😊
Even though it’s been a few years since I’ve developed software, I would say that having a programming background certainly helped me with my DeepRacer experience. AWS provided several resources to review regarding DeepRacer and being able to traverse through the example code was beneficial.
After reviewing some of the AWS DeepRacer learning resources, I used one of the example programs to get a feel for the simulation environment. We had to provide parameters that corresponded to the training environment, hyperparameters (7 different categories), action space (wheel steering angles and speed (minimum and maximum), and a training algorithm which is a software “reward” function.
The reward function is Python code which provides immediate feedback (as a reward or penalty score) when your vehicle moves from one position on the track to a new position. The function’s purpose is to encourage the vehicle to make moves along the track to reach a destination quickly without an accident or infraction. A positive move earns a higher score for the action. An illegal or wasteful move earns a lower score.
I spent most of my time tweaking the reward function. This function has approximately 22 parameters that you can embed in your code to improve performance. Each of these 22 parameters in combination with the 7 hyperparameters, combined with the action space options provided a vast number of options that we could modify to “hopefully” improve our performance. For example, you could combine 1,2, or 7 hyperparameters in conjunction with 1 to 22 reward function modifications. I found out that in some cases changes made it worse, some made it better, or some did not impact the car’s performance at all.
This became very time-consuming because once you make changes then you must select a time for the program to “train” itself based on your program and parameter modifications. Typically, I ran the program for either 30 minutes, 60 minutes, and eventually 120 minutes. These are long times to wait and find out if your changes had any impact on the car’s performance. However, the theme that all of us (my fellow racers) had to adhere to was trial and error. So true!!
You have the option to see how many laps on the simulator you want your car to run to test the model. I chose 3 laps. The first time through my car crashed on all three laps. Not good!! I quickly found out that I had some work to do.
After going through a few modifications … I finally had success!! I made it through each of the laps without crashing. However, it took me an average of 49 seconds per lap. I heard that the top racers were averaging a speed of 7 seconds per lap. At that point, I realized that my view of success was just a moral victory and that my car speed was “slow as molasses”.
Obviously, debugging my program was the next challenge. I had to learn about each of the variables and went through the process of analyzing what modifications could make a difference. You had to be judicious because we did not have unlimited AWS time to train our cars.
I went through the process of making changes through trial and error. In some cases, I was reducing the speed from 1.5 to 1.3 and that made a difference between success and failure. Additionally, running the program from 30 minutes to 1 hour or from 1 to 2 hours made a difference. Although sometimes it was a positive impact and other times it was negative.
It was interesting how my goals changed during this process. First, it was just getting a model to make it through 3 trials successfully instead of going off the track. The next objective was trying to make it through the simulator in under 30 seconds per lap. However, with each minor success, I began to get more driven and competitive to achieve a better result.
Leading up to the race, Acuity put together a DeepRacer helpdesk with office hours during the week. Having the helpdesk was beneficial because Jesse and Obi were able to provide guidance to the racers, but it was also encouraging to hear that other racers were experiencing some of the same trials and tribulations as I had. Additionally, we were able to provide assistance to each other on occasion.
Eventually, it was nice to see that I could improve my performance to 20 seconds per lap. Then I tried more tweaks and failure set in. Yes, I could improve my time to 15 seconds per lap but that included going off track for 2 of 3 laps. Back to square one again. Hopefully, you can understand how this trial-and-error process could quickly change from “the thrill of victory to the agony of defeat.”
This process became rather addicting because I got to the point where I wanted to debug and modify all the variables/parameters. However, I had to realize that it would take too much time and that I needed to stop and submit my model for race day. Success had been achieved!! An average of 14 seconds per lap on the simulator. Not bad for originally starting at 49 seconds per lap.
Race Day …. another interesting experience. We had the option of using the simulator or an iPad to control the speed of our car while using the model that we previously submitted. I chose the iPad option, but I quickly found out that trying to control the speed of the car was more challenging than using the simulator.
Juliann Winn, as part of the pit crew, had the daunting task of moving my car back on the track after it crashed… and crashed…and crashed. Since this was a race, I had the perspective of “go bold or go home” meaning try to go as fast as possible as often as you can. Juliann will tell you that my strategy did not work too well. I think I provided plenty of cardio for her trying to chase my car wherever it went when crashing outside of the track. Sometimes it even seemed to have a mind of its own. (I am sure it had nothing to do with my driving skills 😊) Talk about frustration, Juliann even kicked my car on the last lap!! 😂😂How rude!! Thanks for your help, Juliann!!
Let’s just say I wasn’t in the running for any of the grand prizes, but I made it through and was better for the experience. Lesson learned …. the next time I get involved with DeepRacer, I will stick with the simulator option instead of using the iPad.
Besides having the opportunity to dust off my programming skills, I also appreciated learning about the concept of “Machine Learning”. I had some familiarity with Artificial Intelligence but not ML. Through this process, I was able to learn about the concept of ML and am interested in learning more. Mission accomplished!!
