Lunarium 2: Machine Learning Agent Experiment Day 2

 Machine Learning Hummingbird agent
 Scene set up and script walkthrough to understand essential 
Components that make up the machine learning set up and functionality.  
Ray set up, down and front observes objects in the scene
Similar to Lidar 
Rays are set away from beak to avoid false detection 

Environment boxed in to facilitate bird training 

Nectar Full Flower 

On Empty, Flower changes colour 
Download and install Anaconda 
Creating a new environment 
Python 3.7 was incorrect : use Python 3.8 (64 bit) 
Download latest update from pip – use:
pip install six==1.10.0
$ pip install cherrypy==11.0.0
Fix Courtesy of  Pradyuns
;conta activatr ml-agents -3.0 activates this environment 
Pypi is deprecated 
Set up 8 Training Environments identical to each other.
Ensure Hummingbird is spelt correctly in behaviour 
Use this yml file instead 
behaviors:
  Hummingbird:
    trainer_type: ppo
    hyperparameters:
      batch_size: 2048
      buffer_size: 20480
      learning_rate: 0.0003
      beta: 0.005
      epsilon: 0.2
      lambd: 0.95
      num_epoch: 3
      learning_rate_schedule: linear
    network_settings:
      normalize: false
      hidden_units: 256
      num_layers: 2
      vis_encode_type: simple
    reward_signals:
      extrinsic:
        gamma: 0.99
        strength: 1.0
    keep_checkpoints: 5
    checkpoint_interval: 500000
    max_steps: 5000000
    time_horizon: 128
    summary_freq: 10000
    threaded: true
Yml file courtesy of Point_Nemo on Unity Github
When correct the unity logo will be displayed. 
One you see go to Unity and press play, you should do this immediately. 
If correct you will begin to see data downloading from the hummingbird 
It will take time to see it but the bird will begin to learn about its environment and the flowers within it as it starts to look for nectar. 
Use tensorboard—logdir results instead 
Represents the same data as a graph for ease of reference 
Here it begins to show the hummingbird learning how to successfully locate the nectar.
 
There is a lot of information here ! 
And even more here !
The hummingbird is successfully drinking the flowers dry of nectar as they turn from pink to purple and move on. 
This is where the data is stored on the drive. 
I have left the the training to run to continue the learning process so that I can begin the next part tomorrow. 
Unity Repository 

Leave a Reply