UNCOVER PROJECT

UNsighted COmputer Vision

Introduction

UNCOVer is a project to help sightless person to see objects and tell it's position, describe pointed object , read a text, analyze and desribe and image. In this github repository you will found the needed source codes to run the program on your own device.

Things that are needed

Azure Account

You need an account to access Azure Cognitive Services.

Tools and services that are used

Azure Vision Cognitive Services

Extract rich information from images to categorize and process visual data—and perform machine‑assisted moderaon of images.

Azure Speech Services

Swiftly convert audio to text for natural responsiveness. The Speech to Text and Text to Speech API is part of the Speech services.

Azure Custom Vision

Azure Custom Vision are used for to train a model of index finger. UNCOVer will use this custom vision to detect index finger of the user and decide what object is pointed using our algorithm

Python

Python is used for simplicity, versality, and cross‑plaorm compability. Azure Cognive Services supports Python.

Raspbian OS (Optional - If you don't use Raspberry Pi, use any OS you need.)

Raspbian OS is the default OS of Raspberry Pi. Raspbian is based on linux. The OS will offer flexibility in developing the software.

Thonny IDE

IDE to code and debug in Raspberry Pi 3B+

Hardware that are needed

• Raspberry Pi 3B+
• Raspberry Battery Pack
• Camera Module
• Earphone
• Microphone

Depedencies that are needed

Install the following dependencies via pip:

  
  pip install tensorflow
  pip install numpy
  pip install scipy
  pip install pillow
  pip install matplotlib
  pip install h5py
  pip install keras
  pip install azure-cognitiveservices-speech
  pip install pyglet
  https://github.com/OlafenwaMoses/ImageAI/releases/download/2.0.2/imageai-2.0.2-py3-none-any.whl
  -
  pip install mutagen
  pip install Pillow
  pip install requests
  pip install picamera
  pip install pyAudio
  pip install playsound
  

Documentations & Refences:

Our UNCOVer Project Website

https://uncover-project.github.io

A Quickstart guide on using Azure Speech Services on Python

https://docs.microsoft.com/en-us/azure/cognitive-services/speech-service/quickstart-python

A Quickstart guide on using Azure Text-to-Speech on Python

https://docs.microsoft.com/en-us/azure/cognitive-services/speech-service/quickstart-python-text-to-speech

A Quickstart guide on using Azure Custom Vision

https://azure.microsoft.com/en-us/services/cognitive-services/custom-vision-service/