anti-scaling (1/N) problems

Imagine you have a fixed sized resource. Imagine you can completely consume that resource from 1 client.

Now make this two clients, and completely consume the resource. Which is of fixed size. Each client will get 1/2 (on average) of the resource.

Now make this four clients, and completely consume the resource. Which is of fixed size. Each client will get 1/4 (on average) of the resource.

Don’tcha just love that anti-scaling behavior?

I don’t have a good answer on how to tackle this at the moment. Expanding the resource pool will cost lots of money. My current thoughts (not well formed, so not good) are to break the resource up into smaller individual resources. This might work, but I need to think on how to do this in a way that does (almost) shared nothing so it can scale.

Ugh.

Viewed 29151 times by 5726 viewers

2 thoughts on “anti-scaling (1/N) problems

  1. What kind of resource is this? If it’s capacity, then there are quota mechanisms at different levels. If it’s network or I/O bandwidth there are fewer options, but some OSes still offer some help – e.g. cgroups in RHEL. The problem is that these low-level mechanisms are often poorly (if at all) integrated with the higher-level pieces providing access to those resources, and the distributed coordination problem – multiple producers, multiple consumers, spurious “resource exceeded” failures if load is not perfectly distributed – remains unsolved. FWIW, I’m thinking of taking a tab at this in my own work, but the results won’t be available for a while. If you have ideas, I’d appreciate the help.

    Disclaimer: I work at Red Hat, blah blah.

  2. @Jeff

    In this case its IOPs (looking at them as a fixed size resource). With processes that can completely swamp them with a single client.

    Add in a second client, and the size of the resource pool hasn’t changed, but the average available IOP pool to either client (assuming 100% duty cycle) is now 1/2.

    I am starting to think of complex (interleaved) schedulers and other things. Not sure if what I am thinking makes sense (not enough coffee).

    I can fill you in on this offline if you wish. It is something we want/need to work on.

Comments are closed.