Exploring HTTPS vs. WebSocket for Real-Time Model Inference in Machine Learning Applications

Introduction

Online communication within an application can significantly impact its performance, scalability, and overall user experience. Whether you're building a real-time recommendation system, a live object detection service, or a secure data processing pipeline, choosing a communication protocol is crucial. HTTPS and WebSocket are two primary protocols that serve different purposes.

In this blog, we will explore both protocols characteristics and use-cases.

What is HTTPS?

HTTPS (Hypertext Transfer Protocol Secure) is a secure version of HTTP designed to provide encrypted communication and secure data transfer between clients and servers. It uses protocols like TLS (Transport Layer Security) to ensure the data's confidentiality and integrity during transmission.

HTTPS is often used in machine learning applications to securely transmit data between clients (web applications or mobile apps) and servers hosting machine learning models. For example, when a user submits an image for classification or a text snippet for sentiment analysis, HTTPS ensures that this data is securely transmitted to the server where the machine learning model processes it.

What is WebSocket?

WebSocket is a protocol designed for real-time, bidirectional communication between a client and a server over a single, persistent connection. Unlike HTTPS, which operates on a request-response model, WebSocket allows the client and server to send and receive messages independently once the connection is established.

This makes WebSocket particularly suitable for applications requiring continuous data exchange and real-time interaction, such as live updates, real-time model inference, and collaborative machine learning environments. For instance, in a live object detection system, WebSocket can continuously stream video frames to the server for real-time processing and receive detection results instantaneously.

How does it work in Machine Learning

These communication protocols play roles in machine learning, especially in deploying and serving models, real-time data processing, and ensuring secure data transmission. Here's an overview of how they contribute:

1. HTTPS is critical for securely serving machine learning models through APIs. It encrypts requests and responses, ensuring that data such as client features and model predictions are secured against unauthorized access. This is especially important in applications where the data involves personal or sensitive information (like medical or financial records) that must be protected under compliance standards.

2. WebSockets provide a method for real-time model inference in scenarios requiring immediate processing, such as processing the input images of Autonomous Vehicles or Live Sports Analytics. Maintaining a persistent connection can stream data directly to a model, which processes and returns predictions without the overhead of establishing a new connection for each request.

Advantages and Disadvantages of HTTPS & WebSocket

Both communication protocol serves different purposes. Let's have a look at the advantages and disadvantages:

Advantages:

Disadvantages:

Machine Learning use-cases for using Real-Time Model Inference

Real-time model inference provides instant predictions and decisions, enabling timely and effective measures. This is crucial in use-cases that require immediate action.

Autonomous Vehicles

Autonomous vehicles rely on real-time model inference to navigate and make driving decisions. Machine learning models process data from various sensors (e.g., cameras, LiDAR) to recognize objects, predict the actions of other vehicles and pedestrians, and control the vehicle accordingly.

Video Processing

Real-time model inference is used to process video streams and detect specific objects as they appear in the video. This capability is critical in various applications such as security surveillance, automated retail checkouts, live event monitoring, and content moderation. Machine learning models analyze each frame of the video to identify and track objects, enabling immediate actions based on the detected objects.

Experiment with HTTPS and WebSocket

We have conducted two experiments, simulating a video processing use case in which a video is processed on the client side, and frames are sent to the server for further processing.

The video consists of a total of 573 frames. We utilized the image captioning model Salesforce/blip-image-captioning-large to generate captions, which were then returned to the client.

In both experiments, we observed that HTTPS outperformed WebSocket for video processing.  For the initial experiment, HTTPS processed the task in just 123.04 seconds, while WebSocket lagged slightly behind at 127.36 seconds. In the second experiment, HTTPS completed the task in 21.82 seconds, compared to WebSocket's time of 28.66 seconds.

Experiment 1:

In this initial experiment, we processed the video on the client side and sent each frame individually to the server using both HTTPS and WebSocket protocols.

Here is our observation:

Experiment 2:

In the second experiment, we processed the video on the client side and sent the frames in batches of 16 to the server.

Here is our observation:
‍

Conclusion

HTTPS and WebSocket are essential communication protocols in machine learning with distinct strengths. HTTPS is key for secure data transmission and crucial when dealing with personal or sensitive data. Meanwhile, WebSocket excels in cases requiring real-time interaction and low-latency responses.

The choice between HTTPS and WebSocket should be guided by the specific needs and constraints of the application. Regarding real-time model inference, both protocols have demonstrated effectiveness,

‍