Added Drone Navigation detection using RL techniques

[Panchadip-128]

Pull Request for DL-Simplified 💡

Issue Title :Added Drone Navigation detection using RL techniques

• Info about the related issue (Aim of the project) : Drone Navigation detection using Reinforcement Learning techniques
• Name: Panchadip Bhattacharjee
• GitHub ID: Panchadip-128
• Email ID: panchadip128@gmail.com
• Idenitfy yourself: (Mention in which program you are contributing in. Eg. For a JWOC 2022 participant it's, JWOC Participant) GSSoC Extd Contributor 2024

Closes: #927

Describe the add-ons or changes you've made 📃

Environment:

Grid Structure: The environment is represented as a grid where each cell can be a free space, an obstacle, or the target.
States: The state of the environment is defined by the drone's current position within the grid.
Action Space: The drone can perform four basic actions: move up, down, left, or right.
Reward System:

The drone receives rewards based on its actions:
Positive Reward: Reaching the target yields a reward of +10.
Negative Reward: Colliding with obstacles results in a penalty of -10.
Step Penalty: For each step taken without reaching the target, the drone incurs a penalty of -1, encouraging it to find the shortest path.
Methods:

Reset: Initializes the environment and sets the drone's starting position and target.
Step: Executes an action, updates the drone’s state, calculates the reward, and checks if the episode is done.
Render: Visualizes the current state of the environment, showing the drone, target, and obstacles.
Testing and Validation:

The environment is tested to ensure proper functionality, including state initialization, reward calculation, and collision handling.
The project may also involve training a reinforcement learning agent to optimize its navigation strategies within the environment.
Applications:

This drone simulation could serve as a foundation for developing real-world drone navigation systems, enabling applications in delivery services, surveillance, search and rescue missions, and other autonomous navigation scenarios.

Type of change ☑️

What sort of change have you made:

Example how to mark a checkbox:-

• My code follows the code style of this project.
  -->
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Code style update (formatting, local variables)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested? ⚙️

Describe how it has been tested
Describe how have you verified the changes made

Unit Testing:

Functionality Checks: Each method in the environment (e.g., reset, step, render) was individually tested to ensure they perform as expected. For instance, verifying that the reset method correctly initializes the drone's position and the target.
Boundary Conditions: Tests were conducted to assess the handling of edge cases, such as attempting to move the drone outside the grid boundaries or into obstacles.
Simulation Testing:

Environment Behavior: The simulation was run multiple times to observe the drone's behavior in different scenarios. This included checking if the drone avoids obstacles and successfully reaches the target.
Reward Structure: The reward system was validated by simulating various paths and ensuring that rewards and penalties were applied correctly according to the defined rules (e.g., receiving +10 upon reaching the target, -10 for collisions).
Agent Training and Evaluation:

Reinforcement Learning Training: The drone was trained using reinforcement learning algorithms (such as Q-learning). During training, performance metrics like the number of episodes needed to learn the optimal path were tracked.
Performance Metrics: The training progress was evaluated using metrics such as average reward per episode, success rate in reaching the target, and number of steps taken, helping to verify the learning effectiveness of the agent.
Visualization:

Render Output: The render method was used to visualize the drone's path during training and testing. This allowed for qualitative analysis, confirming that the drone was making logical moves towards the target while avoiding obstacles.
Logging and Monitoring:

Logging Information: The simulation included logging mechanisms to record actions taken, rewards received, and the state transitions. This data was analyzed post-simulation to identify patterns and assess if the agent was improving over time.
Debugging: During testing, debugging tools were used to identify any anomalies or unexpected behaviors in the drone's actions and decision-making processes.
Verification of Testing
Data Analysis: The logs generated during simulations were analyzed to ensure that the agent's learning was aligned with expected outcomes. Statistical methods were applied to assess the consistency of results across multiple training sessions.
Reproducibility: Tests were repeated under the same conditions to verify that the results were consistent, ensuring reliability in the testing process.
Peer Review: The project underwent peer reviews, where findings were discussed and tested independently by team members, providing additional verification and insights into potential improvements.

Checklist: ☑️

Example how to mark a checkbox:-

• My code follows the code style of this project.
  -->
• My code follows the guidelines of this project.
• I have performed a self-review of my own code.
• I have commented my code, particularly wherever it was hard to understand.
• I have made corresponding changes to the documentation.
• My changes generate no new warnings.
• I have added things that prove my fix is effective or that my feature works.
• Any dependent changes have been merged and published in downstream modules.

[github-actions]

Our team will soon review your PR. Thanks @Panchadip-128 :)

[Panchadip-128]

@abhisheks008 Please review once in case of any errors, Thanks

[abhisheks008]

@Panchadip-128 fill up the details first before pushing any pull request.

[Panchadip-128]

@abhisheks008 Done , Please review in case of any errors.

[abhisheks008]

1. You have created two project folders with the same name. Create a single project folder named as Drone Navigation Detection using DL, and put your files inside the project folder accordingly.
2. Follow the below project structure for putting your files,

Project Folder
|- Dataset
   |- dataset.csv (dataset used for the particula project)
   |- README.md (brief about the dataset)
|- Images
   |- img1.png
   |- img2.png
   |- img3.png
|- Model
   |- project_folder.ipynb
   |- README.md
|- requirements.txt

I'll be online this weekend, so please try to finish this task by this week.

@Panchadip-128