The explosion of smart devices, high-bandwidth networks like 5G, and the need to capture and discern data closer to the source have led to the need for a unified edge-analytics application development framework. Devices at the edge, however, are diverse, with each providing their own optimized framework. This makes application portability a big challenge.
What is needed is a framework with cross-platform support that hides the complexities of underlying hardware and allows streaming applications to be transportable across vendors. The framework or runtime must also allow application pipelines to be distributed across hardware vendors, so application developers can choose the right hardware.
Such a framework would provide a unified way to develop and deploy edge analytics applications, regardless of hardware or vendor. It would also make it easier to port applications to different vendors and to distribute application pipelines across vendors, allowing application developers to focus on developing application logic, rather than worrying about underlying hardware or vendor-specific APIs.
To achieve this goal, Megh is enhancing cross-platform support of its Nimble Application Framework, focused on enabling development of edge analytics applications, with the following key features.
REST and WebSocket APIs
Nimble’s REST and WebSocket APIs provide a well-defined abstraction for managing high-level analytics pipelines, while also allowing fine-grained control over how those pipelines are executed across a range of available hardware resources. Open integration is a pillar of Megh’s Open Analytics philosophy, which is why we have chosen to adopt open web standards for inter-service communication.
Secure data transmission (encryption + authentication)
Nimble supports distributed edge computing out of the box, thus encryption and authentication is required for securely transmitting data from edge to cloud. Edge devices must undergo a provisioning process to connect and authenticate with networking services running in the cloud. Industry standard solutions (such as OpenVPN or ZeroTier) are used to create an encrypted virtual private network between edge devices and servers in the cloud. For direct encryption of Nimble’s network traffic, a reverse proxy (such as Nginx) can be used to encapsulate HTTP traffic in a secure HTTPS connection. However, it is recommended to secure the network itself instead of on a per-application basis.
Resource and execution management
Nimble handles the complex task of managing and distributing execution work efficiently across hardware resources. At its core is an advanced thread-pooled task scheduling engine that takes advantage of a pipeline element’s metadata to know when the element can be processed in parallel. The design of Nimble’s execution engine enables optimized pipeline data throughput and minimal latency. SDK users can focus on the application logic of their custom pipeline elements instead of worrying about low-level resource and execution management.
Software development kit
Nimble empowers developers to create custom pipelines by providing a comprehensive SDK for incorporating new models and accelerating them across devices. The SDK also provides a flexible pipeline element interface that gives developers the ability to create simple and complex analytics. These models and analytics can be written in a variety of programming languages. Support for popular open source libraries is included in the base SDK and others can be easily added.
Hardware acceleration support
Nimble also includes an API for hardware accelerators. With this API, developers can easily integrate platform hardware accelerators with Nimble and expose them as a pipeline element. The plugin framework is built from the ground up to maximize performance. Native code improves execution time and our zero-copy architecture removes memory bottlenecks. Nimble includes plugins for CPU (OpenVINO, ONNX), iGPU (OpenVINO), and GPU (Triton) out of the box. We are also working with several ASIC hardware partners, including Qualcomm, to support their platforms.
In summary, Nimble is a powerful framework that enables the development of edge analytics. It provides a unified way to develop and deploy edge analytics applications, regardless of hardware or vendor. It also makes it easier to port applications to different vendors and to distribute application pipelines across vendors, allowing application developers to focus on application logic. Important features include REST and WebSocket APIs, secure data transmission, resource and execution management, and a software development kit for models, pipeline elements, and hardware acceleration.
For more information, see https://docs.megh.com/vas-sdk/intro.