How to calculate server configuration accroding to request per second

Suppose I want to build a server. My estimated request in my sever is 50,000 per second. How many cores, rams, network nodes do I need to build that server.

Lets think, you have a cloud platform where there will be controller/compute nodes, networking
nodes, and storage nodes.

  1. Now, for an application, you need to serve around 50,000 requests per second [you can
    think about the scenario when our education board results are published i.e ssc, hsc, jsc, psc,
    etc]. How many compute/networking, storage nodes will be needed? Must write the hardware
    configurations also as request handling will vary according to the server configuration.
  2. Do the same task for handling 100,000 requests per second.
  3. Do the same task for handling 250,000 requests per second.
  4. Make a cloud computer based on these requirements.
    Input - how many requests, and server configurations
    Output – how many compute nodes, storage nodes, and networking nodes will be needed.

Each software solution should come with recommendations about that. Either those are provided by vendor, or you measure them in the test environment. Best actually is to do both ask vendor end measure them independently. And be skeptical about the vendor’s information.

All of them are building “scalable” solutions, most of them fail at it.

Also if you just want to explore things theoretically - start with CAP theorem:

Trust but verify.

The use case you’re mentioning require serving static pages; publish once serve many. Strictly speaking requires parsing the request and streaming back the results from caches in ram. You might also need to serve some static visual page elements and some javascript for the browser to put together a page.

If the are small (<1MB), I’d use 10k qps per core as a rule of thumb; that would be normal for any reasonable c/c++/go perhaps even Java webserver perhaps these days if you tune the jvm a little bit.

I’d be more worried about your network setup (who provides it and at what cost).

Also I’d be worried if web developers would do something stupid like design those stupid pagination systems that return 25 results per page or do sql database queries in the critical path of those 250kQPS. If you need keyvalue serving stick the dataset into a memcached instance on server startup, count the number memcached requests per page. As a rule of thumb, if it’s more than 10 you should reorganize.

As for cloud, I’d say do not use, they’ll charge you per load balancing request and nickle and dime you to death on bandwidth; (at 250kQPS you’ll run out of money very quickly). Go to (or contact, don’t go anywhere) your local university IT dept staff that take care of the server infrastructure. Talk to them, they should be able to hook you up with some VMs or with access to a kubernetes cluster you can use. They can also recommend what makes sense to do to avoid a thundering herd effect.


Lastly, if you’re desparate and have a gig of data in something like csv files, and want to distribute it among a bunch of people, students and such, and are alone with zero budget. Pack it into a torrent file, announce the torrent on a public tracker and ask for help in reddit. …(if the other people you’re working with have only little experience with these 250kQPS web things, prepare a plan b, get ready to go this anyway)


On the other hand if this is all a theoretical exercise, lookup site reliability engineering books from Google, read up on NALSD interviews Google does. SREs at places like Google Microsoft Facebook Amazon Netflix Apple and other places that run cloud services and/or clouds ; like to interview people for SRE roles out of smaller shops like e.g. your local university it dept that has a few dozen apps, maybe 100k/200k users and sometimes does high qps stuff like you’re asking. Google in particular does this relatively methodically, which makes sense considering how large of a fraction of the world’s compute and storage and network capacity those people end up being responsible if hired.

1 Like