The internet brags an immense range of resources facilitated on various servers. For you to get to these assets, your browser should have the option to send a request to the servers and show the resource response for you. HTTP (Hypertext Transfer Protocol), is the basic organization that is utilized to structure requests and responses for compelling correspondence between a server and a client. The message that is sent by a client to a server is the thing that is known as an HTTP request. At the point when these inquired requests are being sent, the client can utilize different methods.

Along these lines, HTTP demand techniques are the resources that demonstrate the particular wanted activity to be performed on a given asset. Every strategy executes an unmistakable semantic. However, there are some standard highlights shared by the different HTTP request techniques.

HTTP Request Methods

HTTP characterizes a bunch of request techniques to show the ideal activity to be performed for a given asset. Even though they can likewise be things, these request methods are now and then alluded to as HTTP action words. Every one of them executes an alternate semantic. However, some regular highlights are shared by a group of these, like a method of request that can be cacheable, safe, or idempotent.

  1. POST:
    The method POST is utilized to present an entity to the predefined asset, frequently causing an adjustment in state or results on the server.
  2. GET:
    The GET method demands a portrayal of the predefined asset. Requests utilizing GET should just recover information.
  3. PUT:
    The method PUT replaces all current portrayals of the objective asset with the payload of the request.
  4. HEAD:
    The HEAD technique requests a response that is indistinguishable from that of a GET request, yet without the body of the response.
  5. CONNECT:
    The CONNECT method sets up a passage to the server that is distinguished by the objective resource.
  6. DELETE:
    The DELETE method erases the predetermined asset.
  7. TRACE:
    The TRACE technique plays out a message testing of loop-back along the way to the objective resources.
  8. OPTIONS:
    The OPTIONS technique is utilized to depict the correspondence choices for the target asset.
  9. PATCH:
    The PATCH technique is utilized to apply halfway changes to a resource.

How does HTTP Request work?

HTTP represents Hypertext Transfer Protocol and is utilized to structure requests and responses over the web. HTTP expects information to be moved to start with one point then onto the next network.

How about we assume, you type an address, for example, www.abcxyz.com into your browser, you are telling it to open a TCP channel to the worker that reacts to that URL (Uniform Resource Locator). A URL resembles your residence or telephone number since it portrays how to contact you on a browser. In the present scenario, your PC, which is making the request, is known as the client. The URL you are mentioning is the location that has a place with the server.

The exchange of assets happens utilizing TCP (Transmission Control Protocol). In essence, TCP deals with the channels between the server and your browser. TCP is utilized to oversee numerous sorts of web connections in which one computer or device needs to send something to another. HTTP is the language of commands that the devices on the two sides of the connection should continue to convey.

HTTP Requests in Parallel:

Have you ever come across a situation where you need parallel HTTP requests at a time? Most browsers only open 2 HTTP connections to one domain at a time.

Understanding the Serial & Parallel Requests:

Usual browsing follows the traditional serial requests in which a client requests a query or searches a URL on the web by sending an HTTP request. The server fetches the data and sends it back to the client by an HTTP response. In this procedure, reuse of an open connection takes place with no multiplexing in a serial manner. It is also referred to as a keep-alive HTTP connection that has an open connection by a client and it reutilizes it to send back a response to the client machine for whatever has been requested to fetch in an HTTP request.

Serial HTTP Request

Serial HTTP Requests

On the contrary, the advancement in technological spheres has emphasized the use of parallel HTTP requests to save time as it reduces the time complexity up to a greater extent. In parallel HTTP request, more than one requests are sent by a client to the server and server processes the request and response back in parallel, making this technique more time-efficient and reducing resource utilization as well.

Parallel HTTP Request

Parallel HTTP Requests

Pre-requisites for Establishing Parallel HTTP Connections

  • Reduce the file size. The smaller the size, the better the response time would be.
  • Remove unnecessary files of CSS and if supported, then use CSS sprites.
  • Consider inline images using data URL.
HTTP Connections

Conducting an Examination of Parallel HTTP Requests

In the wake of lessening the size of the data file to several bytes, while setting up numerous HTTP connections simultaneously, there needs extra asset utilization. For every HTTP, there is haphazardness in the DNS (Domain Name Server) inquiry time and the foundation time of the TCP connection, so it will make a more noteworthy possibility of expanding the time utilization of a specific HTTP altogether while mentioning assets simultaneously. The most limited load time noticed is (16ms) and the longest load time is (32ms). The time distinction can be diminished up to 72% if nearly a more modest file size is utilized. On account of the figuring time depends on the finishing of stacking all assets, making different HTTP requests simultaneously will prompt more noteworthy time non-consistency and vulnerability, which causes it frequently slower than making one HTTP request to stack the consolidated resources.

Examine HTTP Requests

Time Comparison Analysis of Serial vs Parallel Requests

Serial HTTP Requests Time (Processing Two Requests)Parallel HTTP Requests Time (Processing Two Requests)
Time of TCP Setup (Network) = 40 msTime of TCP Setup (Network) = 40 ms
Request1 Time = 20 msRequests Time = 20 ms
Processing Time = 40 msProcessing Time = 40 ms
Response1 Time = 20 msResponses Time = 20 ms
Request2 Time = 20 ms
Processing Time = 40 msTotal time for two requests & two responses equals 120 ms.
Response2 Time = 20 ms
Total time for two requests & two responses equals 200 ms.

For both Serial and Parallel HTTP Requests Time,
Network Overhead Time = 60%
Improvement = 50%

How to do Parallel HTTP Requests on the Web

It has been tested several times in many studies and found that serial HTTP requests mostly work without any failure or disablement. Yet we want to devise a method to make the parallel requests to be conducted successfully same as serial does. The following are to be considered for parallel HTTP requests.

  • Setting parameters for parallel requests
  • Timeout Management
  • Priority Management
  • Persistent & non-persistent connection
  • Performance Evaluation

HTTP requests can transfer the requests and response back in a high-performant way.

Setting parameters for parallel requests:

First of all, the main task is to finalize the parameters to conduct a test of parallel HTTP requests like timeout management of the request-response time, the priority of the requests that are being sent in parallel, connection to be determined as persistent or non-persistent connection, etc.

Timeout Management:

For conducting an investigatory study, we need to consider small chunks of data within a certain range. The time estimation will be based on that the maximum RTT is 200ms. The data transfer duration is to be monitored and timeout is calculated on average durations.

Priority Management:

Priority management of the parallel HTTP requests plays a vital role as the responses from the server will be processed and supplied by that. The priority order of data requests ensures the successful transmission of the response. The in-order throughput of multiple HTTP requests gets stabilizes when prioritized accordingly as it assures the in-time response delivery to the client.

Persistent & Non-Persistent Connection:

A persistent connection (HTTP industrious association) is an organization correspondence channel that stays open for additional HTTP requests and responses as opposed to shutting after an exchange. While a non-persistent connection takes the time of connection for 2RTT + transmission time of a file. It takes the principal RTT (Round Trip Time) to build up the connection between the client and the server. After the client gets the item in non-persistent, the connection is quickly shut down.

Performance Evaluation:

The performance evaluation of the parallel HTTP requests can be done in two phases, i.e., TCP friendliness & data robustness. Firstly, the friendliness of the TCP connection of the client-driven request to be evaluated. Secondly, the robustness of the response is to be determined concerning the delay and packet loss.

Parallel HTTP Requests with javascript and Crawlbase (formerly ProxyCrawl)’s Scraper API

For the sake of keeping things short, we will use Crawlbase (formerly ProxyCrawl)’s Scraper API to do parallel HTTP requests.

We will use Visual Studio Code as it is one of the most popular and accessible editor that can be used on the majority of operating systems.

Before we dive into coding, let us prepare our project structure and be sure to install all prerequisites.

Once done, let us start writing our code in the first .js file that we have created (Start.js). Our first two lines will declare all the constants and require the necessary API class in this project.

1
const { ScraperAPI } = require('proxycrawl');

The next line will be important as it will hold the value of your Crawlbase (formerly ProxyCrawl) token and the websites to do parallel API calls:

1
2
3
4
5
const api = new ScraperAPI({ token: 'YOUR_TOKEN' });

const promise1 = api.get('https://monday.com/');
const promise2 = api.get('https://www.codecademy.com/');
const promise3 = api.get('https://jsonplaceholder.typicode.com/');

Now, we can write a simple API call based on the Crawlbase (formerly ProxyCrawl) library to scrape a website of your choice and then print an output in the console with based on

1
2
3
4
5
6
7
8
9
10
11
12
13
const { ScraperAPI } = require('proxycrawl');

const api = new ScraperAPI({ token: 'YOUR_TOKEN' });

const promise1 = api.get('https://monday.com/');
const promise2 = api.get('https://www.codecademy.com/');
const promise3 = api.get('https://jsonplaceholder.typicode.com/');

Promise.all([promise1, promise2, promise3]).then((response) => {
  for (let i = 0; i < response.length; i++) {
    console.log(response[i].url + '\n' + response[i].body + '\n\n');
  }
});

The output will look similar to this in JSON format.

JSON Format 1 JSON Format 2 JSON Format 3 JSON Format 4 JSON Format 5

Conclusion

The parallel HTTP requests need more improvements from research perspectives to get an in-depth insight into the optimization of synchronized performance by the reduced size of data files, utilization of lazy load components, CSS head tags, JSON instead of XML, preferring to GET method unless you need POST method and the use of hardware for the improved and accelerated effects of parallel HTTP requests.