self-driving-car-sim

u install. Of course, that maxed out the GPU and put the fans on high, but it still worked fairly well at a decent resolution and level of detail settings. Just to show you don't need an expensive GPU to play around with this stuff...

information: https://arxiv.org/abs/1604.07316 This is actually a fun CNN to implement - I (and many others) implemented variations of it in the first term on Udacity's Self-Driving Car Engineer Nanodegree. We weren't told to do it this way, but I chose to do so after reviewing the various literature, plus it seemed like a challenge (and it was for me). Udacity supplied a simulator: https://github.com/udacity/self-driving-car-sim ...and we wrote code in Python (Tensorflow and Keras) to train and drive the virtual car. For my part, I had set up my home workstation with CUDA so that Tensorflow would utilize my GPU (a lowly GTX 750 TI SC - though it seems like it might have a similar GPU capability as NVidia's Drive-PX system, based on what I've researched - a Mini-ITX mobo, a PCI-E slot riser, and a GTX 750 would make a decent low-end deep-learning platform for self-driving vehicle experiments, and cost a fraction of what the Drive-PX sells for).

fore with keyboard only and no problems) is that joysticks were really touchy (major fast acceleration on movement of sticks, and no way to really adjust it easily under Linux - which was kinda a disappointment to see - I could do some adjustment, but nothing like what is available on Windows); I found that using a steering wheel controller worked best.

d braking values as well - so we could use them if we wanted to). The simulator had three "cameras" - left, right, and center. You could drive it around the course (using a keyboard, mouse, or joystick/pad), and it would dump these frames to a directory, as well as a labeled dataset of the frame filenames and the various datapoints. So - you had a labeled dataset for training a neural network - input a frame, train it to output the proper values for that frame (steering, brake, throttle). The left and right frames (with other adjustments) could be used to simulate drifting near the left or right side and counter-steer away (to prevent going off the road). Other things we did was to augment the data by adding fake shadows, shifting or skewing the images to simulate turns or whatnot, altering the brightness of the images to simulate glare, etc. So you could get a very large dataset to play with; more than enough to split up into training, test, and validation sets. For myself (and more than a few of the other students), rather than try to implement a custom network to handle the task, we instead implemented a version of NVidia's End-to-End CNN: https://images.nvidia.com/content/tegra/automotive/images/20... Some of the details about the architecture you have to make some educated guesses about, plus we had to scale our images down to reduce the number of input parameters to the network (but if you have the memory, go for it!). I ended up using my GTX 750 ti SC for my TensorFlow CUDA (running everything under Ubuntu 14.04). This has about the equivalent power as NVidias embedded Jetson PX car computer platform (better for running the model on rather than training, but I don't have such a real rig yet). It worked really well. I'd imagine that if you did the same in GTA V, which has more realism, the resulting model could be used as a foundation (transfer learning) to further train a vehicle for real-world driving. I suspect that Udacity's simulator has three cameras because the self-driving vehicle they developed has the same setup (I'm not sure how it is going to work, but the final term project is supposed to be related to the real vehicle in some manner - they claim our code will be ran on the vehicle, but we'll see how that goes). So...you might want to give that a shot! As some advice here, I found that for Udacity's simulator, using a steering wheel controller system (with pedals) was more natural and generated better data than using a keyboard/mouse/joystick. Also, while it sounds like implementing this kind of system is simple, in reality you might find yourself being frustrated by certain aspects (make sure your dataset is "balanced" - I found issues in one of my runs where my model was biased to left-hand turns, and couldn't cope with a right-hand turn, for instance) - but these can lead to interesting and humorous results: I had one model which got confused, and would turn off the road in the simulator and drive on a dirt area; this dirt area was defined, but "edges" consisted of berms and warning signs; other than that it looked the same as the ground. The dirt area wound around then eventually led back out to the track. My model, driving the car, would invariably go off-roading on this section - but despite never having trained in this area, it drove it relatively perfectly! It would follow the dirt around, and get back on the track, at which point when it got to the dirt entrance again, it would do it over. It did this several times, and was fun to watch. Somehow, the CNN model had enough knowledge from on-road driving to handle a simple case of off-road driving as well... So you might want to give this option a shot in the future. I personally have waiting in the wings a machine that I planned to build for running GTA V, based around a GTX 970 - it was going to be a gaming machine, but now I am seriously considering making it my next workstation for machine learning (and perhaps upgrading to a 1080).