REST API, group operations

Background

I am coding a custom API backend, and as part of that end up needing to design that API. Inevitably I aim for REST and thus meet its challenges.

The problem

The design choice I am looking into is how to handle operations over multiple objects. The main case I consider is deletions of sessions, since a hypothetical shared/“shared”(aka. hacked) account will have enough of them that deleting them individually will have significant overhead and delay could be a real issue.
There is the

The solutions

Since REST as a concept is pretty much as old as HTTP there are some traditional solutions to this. The ones I have found (and their properties) are:

Don’t solve it

Users probably won’t use this multiple operation feature and may be confused by it. Just don’t bother and perhaps instead work on minimizing the overhead on small operations.

Advantages:

  • it requires no additional effort
  • keeps the API simple and predictable for users
  • granular error reporting

Disadvantages:

  • performance “loss” (absence of gain) on large user operations
  • potential user frustration if they would benefit from operation grouping

Summary:
Probably the best option for an API where users don’t have that many objects, since there isn’t much performance to gain from grouping unless a lot of objects need changing.
If there is a specific case where you need to support, consider if you could solve it specifically instead of generally (in my case, support deleting all sessions for a user on a separate path).

Allow operations on collections

Since REST APIs are already recommended to handle filtered GET requests on collection paths it would be a reasonable expansion to allow filtered DELETE requests on the same. Since this can be translated into a single database query using those filters (at least in my case) it is no more prone to errors, but this may vary depending on use case.

Advantages:

  • code symmetry (GET / DELETE)
  • predictable
  • good performance (the best if you collapse it into a single database query)

Disadvantages:

  • the same cannot easily be done for other VERBS
  • introduces gotcha (DELETE on collection with no filter)
  • less granular error reporting (but unlikely to be needed due to ->)
  • very limited operation grouping (only within a collection path)

Summary:
A reasonable middle way for when users are likely to have more objects and could wish to delete chunks of them, but you don’t anticipate grouped PUT, POST or PATCH requests.
If the limitation against PUT, POST, PATCH frustrates you there are some possible reasonable interpretations on those, just beware of assuming that nothing will change between your requests and the vulnerabilities that could appear if you allow PUT to create/update objects with a user-given id.

Create a specific grouped operation handler

When there doesn’t seem to be any correct and intuitive way for you to group the commands you wish to group you turn to the mightiest of VERBS: POST. The syntaxes that I’ve seen take a json list of operations (path, VERB and body) and returns a list of results for each of those. Some further implement a semi-templating feature that allows using data from one operation in another.

Advantages:

  • unlimited potential: any VERB any grouping anything (you can define a turing complete domain specific language for your API and calculate pi)
  • granular error reporting
  • good performance

Disadvantages:

  • requires significant documentation
  • a lot more code
  • encourages you to duplicate your API (be reasonable, use functions to minimize duplication)
  • DoS risk (depending on features and rate limiting this could use a lot of server side performance)

Summary:
For more complex situations a separate handler could well be warranted, but it is not something to implement on a whim. That said, a minimal implementation (list of independent operations) is not prohibitively difficult to implement and is immediately more powerful than any other option.

I have pretty much settled on the middle option (since I have the filtering code ready and just need to change “SELECT” into “DELETE” in the query) but I’d love for some input on this summary and on what you have chosen in your projects.

I am missing a little bit of context here, but are these users directly accessing a database here. Are you limiting their ability to submit raw queries, IE giving them access to a visual query builder, or are these functions directly tied to buttons?

Generically, the middle option sounds like the best option since you pigeon hole yourself if you are too optimized or not optimized enough. No one likes to rewrite code and if you start bolting on things, it gets ugly quickly.

I would write as free form query option so that you can experiment with it, but not expose it to the general user in the public facing API, or require a signature on the message in order to process it.

I accept a whole bunch of optional filter arguments in the API request, which I then hand into to a prepared query which additionally filters to sessions owned by the user. (boiler-room/sessions.rs at main · sidju/boiler-room · GitHub)

As it is now I only have the backend REST API, but I have started writing a frontend for it (in part to be able to properly test). The plan is reasonable authentication to authorize limited changes to the database, to later be used for other projects. The whole thing is intended as an overgrown boilerplate for future full-stack projects, a boiler-room one might say.

1 Like

Ah. That makes sense then. That is going to be a tall order to fill then. I expect that you will go back and add things to the boilerplate as they are discovered then. In that case, I would say make this as modular as possible. If it is boiler plate, it can be a little bloated if it adds easy of upgradeability later on since no one will be interacting with it directly, per se.

1 Like

I would design this as have each item DELETE be an individual request.

Front-end can request what each one is, then make the delete for each individual, your front-end can then choose to handle how to display each delete etc. (maybe display a pretty list of each one that’s progressing, and remove/strike through/red text each one as it gets deleted, so user gets a visual guide of progress) and you can follow that a request does an all or nothing approach, either everything in the request is deleted or nothing. Doing it with larger list is a bit more difficult and you can’t use a UI way to get around it without having to start messing with sending back a list of what failed (what if one that failed caused a crash or transaction to spin?).

I’d only make group requests if this was still HTTP/1 and these are large id lists, HTTP/2 multiplexing overhead doesn’t matter as much.

This of course assumes in the tens/hundreds of delete requests, I haven’t come across a use-case for millions where it wasn’t a “container” object, and assuming the overhead of searching DB for each request isn’t that high, you’d need to test it with your use-case (e.g. system I am working on now there are 2 use-cases for a list of ids, but they end up being long-running processes and not quick delete requests).

You have a good point, this is probably pre-emptive optimisation.

Whilst this is HTTP/1 there really won’t be that many entries in any list, at the intended scale of the API.