The JeVois software features the following components:
- All software open source (GPL)
- Full Linux operating system runs on the JeVois smart camera's quad-core processor, boots in 5 seconds
- Custom kernel modules for camera sensor & USB streaming run inside the smart camera
- Buildroot framework to easily add software packages and create microSD card image
- JeVois C++17 video capture, processing & streaming framework
- Switch machine vision modules on the fly by changing output resolution
- Download pre-programmed machine vision modules or create your own
- CMake build system
- Full cross-compiler suite (compile all software on your desktop)
- Compile and run the same software on desktop and on JeVois hardware, at the same time (very useful for development & debugging)
- Operating system and vision software all stored on microSD card, hacker-friendly and unbrickable. Smart camera can use microSD to save data.
- Included software libraries (used for the demos seen in the video):
- Neuromorphic algorithms for visual attention & scene understanding
- OpenCV 3.2 machine vision algorithms
- All opencv-contrib modules (object recognition, ArUco, etc)
- ZBar library for barcode & QRcode detection and decoding
- tiny-dnn library for deep convolutional neural networks
- GPU-accelerated image processing using OpenGL ES2.0 shaders
- Support for NEON multimedia processor instructions
- Vlfeat library for visual feature computation
- OF_DIS library for fast motion flow computation
- Eigen3, TBB, OpenMP, etc
The JeVois smart camera runs a full-featured Linux operating system directly on the quad-core processor inside the camera. You can interact with the camera over the USB link and over the serial (UART) link to change its behavior at run time. You can even start the smart camera in debug mode and log into it as you would do with any other Linux computer!
Host system requirements
The JeVois smart camera can work as a standalone computer, with no USB video streaming. In such case, one would usually simply stream commands to an Arduino or similar over the serial port. All you need then is to provide power to the JeVois camera's mini-USB connector.
For video streaming over USB:
Linux (including Raspberry Pi): Works out of the box, no drivers needed, all functionality is available. You can switch between different vision processing modes on the fly and at runtime, by selecting different camera resolutions and frame rates on your host computer. For example, 640x300@60fps may run the visual attention algorithm, while 320x256@60fps may run the QRcode detection algorithm. A configuration file on the MicroSD card establishes the mapping between USB video resolution and frame rate, camera sensor resolution and frame rate, and vision algorithm to run.
Windows, Mac OS X: Streaming works but selecting among different available video resolutions has become increasingly difficult with newer versions of Windows and OS X. It seems that these operating systems do not want to let you choose a camera resolution or frame rate. This choice, however, is how one can switch on the fly between different vision algorithms on the JeVois camera. One workaround is to configure your JeVois smart camera with only one vision algorithm (achieved by commenting out entries for other resolutions in a configuration file on the MicroSD card). Then the host computer will have no choice but to use that one. Also see this documentation page for more information.
Android: The camera is detected but streaming video does not work yet. We are working on this and we suspect this has to do with deviations from the USB video class (UVC) standard on the Android side. For example, a USB packet sniffer has revealed that the Android device insists on querying camera controls that were not declared as being supported by the JeVois hardware.
iOS: The JeVois smart camera, and any other USB camera we have tried, are currently reported as not being a supported device. This may change in the future.
Developing for JeVois
Everything is cross-compiled on a host computer. At present only Linux is supported for compiling JeVois software. JeVois software is written in C++17, with a few low-level algorithms in optimized C. We use CMake to manage the build process. This allows you to build for both your host computer and for the JeVois hardware, both at the same time. This is very useful during development, since you can test your algorithms using any USB webcam and observe the results in a window on your screen. You can also use your host computer to train a deep neural network quickly, and then just load the weights onto your MicroSD card. The JeVois operating system is Linux and its features are managed by buildroot.
Scripts are provided to automate the process:
rebuild-host.sh - compile the entire JeVois software suite natively on a host computer. You can use it with a USB webcam, and it display outputs in a window on your screen.
rebuild-platform.sh - cross-compile the entire JeVois software suite for the smart camera hardware.
jevois-build.sh - flash the cross-compiled results from rebuild-platform.sh to a MicroSD card.
Try it for free
The JeVois software framework can be compiled and run on a standard desktop computer running Ubuntu Linux 16.10 (64-bit). You can download it and try it on your desktop using a regular web cam to provide video inputs. Details are at http://jevois.org.