Integrated software and hardware solution for remote specialist consulting
About the Client
Our client is the head of a company focused on Research and Development in the field of natural and technical sciences. He connected his life with inventions, innovations, and rationalization.
After winning a sci-tech festival, he invested his effort in the development of a new hardware and software solution for the remote specialist consulting. The Client conducted a market analysis, confirmed the relevance of the problem, and decided to test his idea for viability.
Spoiler alert: Subsequently, the Client twice received funding from the Innovation Assistance Fund through collaboration with the Inostudio team.
Client’s Task
The Client's task was to create a solution allowing experts to consult specialists from anywhere in the world in real time.
After a joint discussion of the idea, the entire project was split into several independent components:
constituents
headset design and development
constituents
optimal components selection
constituents
hypothesis testing through mobile and server solution development
Working Headset Prototype
The main idea of the startup was to reduce the business trip costs for a consulting expert engineer as well as reduce the overall repair time and equipment downtime. That is, in order to improve the quality of service and communications, we needed to prepare an effective workplace, a functioning headset prototype for consulted engineers. After analyzing and selecting the necessary components, it was decided to stick to simple solutions without highly specialized components.
Microcomputers, Smartphones and Stereo Cameras
To transmit the image from the headset, a camera had to be used, i.e. a working computer with communication modules, a microprocessor, and an input interface. And we faced a fairly simple choice at that time: we had to choose between primitive and expensive microcomputers or modern smartphones. Intuition and experience said to opt for the latter.
As a result, it turned out that there was no right option at all. Microcomputers were expensive, and smartphones, even the best of the best were not able to transmit a stereo image. And that was extremely important to properly deal with the displayed objects.
Therefore, we started searching for a stereo camera. And paradoxically, the most economical solution was quickly found in China. This is how we and our client came to the technological minimum set which included:
- a smartphone as a microcomputer to transmit and receive signals;
- an external camera to send images to the consultant’s web client;
- an Android mobile application to achieve the synergy between a consultant and a specialist;
- a webRTC application to achieve the synergy between a consultant and a specialist.
Mobile Application for Android
To work with the equipment, our team found the only one open platform, a smartphone with the Android operating system. And in order to debug at the lowest level, to interact with drivers, we chose native development on Android OS.
Web Client for Remote Consulting
The other side of the DAR complex hardware and software solution is a web client that receives an image from a specialist's stereo camera. When the web client and the headset work together, the consulting engineer will be able to:
- send signal-indicators to the required area on the specialist’s smartphone screen for the complex equipment maintenance;
- assist the implementation engineers in operating various equipment or train them from scratch.
It was important for the Client to implement a software solution that could provide any assistance remotely, regardless of the business specifics or its complexity.
Problem Solving
When all the necessary modules and startup equipment were determined, we and the Client moved on to the next stage. It was the implementation of the application and the web client.
01. Mobile Solution
We compiled a list of hypotheses, tested them, discarded the unnecessary requirements, and focused on the main thing, i.e. developing an application that can:
- manage minicamera settings (choose contrast, brightness, and resolution);
- display the video stream full-screen on the smartphone in real time and transfer it to the consultant;
- superimpose elementary graphics over the video stream.
When we made sure that the external camera fulfills all the embedded scenarios, we implemented a stand-alone application for it. As a result, we were able to change the scale, focal length, work in different resolutions with different definitions. Next, we implemented real-time video support and moved on to developing a server-side solution.
02. Server Solution
The development of the web client began with the choice of protocols, so we needed a balance of quality, security and prevalence. We settled on WebRTC, one of the most popular technologies that describes the streaming data transmission such as audio data, video data, and content here and now.
As a result, we developed an organic connection algorithm for the camera, the application and the web client. That is, the camera module receives and broadcasts the image, and the smartphone operating system does the rest. The entire result of the algorithm is transmitted to the consulting engineer’s web client.
Collaboration
Two-year collaboration with the Client is an example of an equal partnership relationship. During the global crisis and pandemic, we had a real collaboration of joint efforts. All parties were interested in the development of the DAR product, in solving the complex technical startup problems.
Everyone took an active part in the project. The Client, for example, took on all the work of designing and developing the headset prototype, as well as the task of optimizing the connecting nodes. Our Client also tested the product and formed the DAR development vector.
In our turn, we, in addition to developing web and mobile solutions, focused on sorting out hypotheses in order to ensure the implementation speed of Research and Development.
Difficulties and Special Features
The development of IT solutions is accompanied by various difficulties. The difficulties complexity and their number depends on the specifics of the business. In this case, our team faced with the non-standard situation as there was no such algorithm as “there is a problem, there is a solution”. Instead, we had to use an experimental approach. We were constantly looking, trying and testing, like pioneers. The thing that makes the complex software and hardware product DAR interesting is that it was required to make the code friends with no-name-hardware. Therefore, the whole path was thorny and filled with quite funny nuances.
Result
The main indicator of the successful work of the Client and Inostudio on the DAR product is getting the funding for the first and second stages. In addition, we gained unique experience in Research and Development and improved our skills in webRTC project work.
Technology
Client’s Feedback on Working with INOSTUDIO
I will definitely contact Inostudio again. This is a team that, having taken up something, will definitely complete it to the very end and will not be stopped by any difficulties. I have never regretted that I came here, I recommend it to everyone!
