A crooked phone photo goes in, a clean straight card image comes out - no GPU, and nothing running when nobody uploads.
It runs on the biggest lambda AWS sells, and the biggest is not the same as enough.
Introduction This is the 3rd article in the series about the image processing pipeline.
In two previous articles we went over the whole image processing pipeline, which process AWS Builder Cards images: From anonymous photo to a published page: An event-driven, AI, image processing pipeline on AWS and also over image classification model running in lambda container, which filters out the uploaded images and filter only valid AWS Builder Card images: Is this even a valid card?
Zero-shot image classification model in a lambda container Today's article is about vision segmentation model, which also runs in a lambda container, but it's role is to process the filtered image - remove background, straighten it and crop it.
Specifically about this part: The problem Somebody photographs an AWS Builder Card lying on their desk and uploads it to my collection.
What arrives is not a card - it is a "photo" of a card.
Tilted and with some background around it.
One lambda has already looked at it before - a small, container based lambda , which filtered it as valid AWS Builder Card.
See this article for more info.
Now we have to clean that picture (straighten, crop, remove background), which is a job of lambda function .
That lambda answers the question: Where exactly is the card in this photo?
The coworking between the two lambdas is simple: after the image is put into S3 and filtered by as a valid one, overtakes and start to processing the image.
Can I run it in the container?
To process the image as describe above, I need an image segmentation model.
The one that goes pixel by pixel and says: "card, background, card, background..." I need it to run on , inside a lambda container from the same reasons as (inside AWS, cheap, serverless).
To find a right model to do the job I need to know the answers to the same questions, as with lambda running : How much CPU and memory does it need?
Where do the weights come from?
What runs at the inference?
The inference In this case, this is the easiest one to answer.
It runs on behind a small library called , which means no and no training framework anywhere in the image.
The other two get a section each further down, and question 1 is where the story is: the model I picked first barely fit at .
Lambda also ties vCPU to memory, so that number decides how fast this thing runs, not just whether it survives.
So this is an article about memory, about the bill, and about the geometry that happens after the model has stopped talking.
One thing before we start, so it does not ambush you later: there are two models in this lambda. finds the card.
Right at the end, a vision model reads the text off the finished image.
But let's start from the beginning The model As an image segmentation model, I decided to go with , loaded through .
The Lite is also the answer to the memory question.
The memory story Remember it works on my machine meme from previous article?
Here is where I really earned it.
What I was working with locally was the model.
It worked fine (on my machine 🤣), so I shipped it.
What I never did was to watch how much memory it was eating while it worked.
I started the container at 4 GB memory.
The first couple of invocations showed me the problem: .
So I went to maximum I could - 10 GB memory.
It worked fine, but not as per : .
That's wonderful 97% of maximum container's capacity.
So yet it works, but then I tested 125 MB image and it crashed.
There is no 12 GB to escape to, no bigger instance type, no flag to ask for more.
With standard <20 MB image 97% of the memory is out of the question.
Because I could not buy more memory, I had to need less of it.
The answer was lighter model from the same family: Setup What happened 4 GB + Dies at - used 10 GB + works, but peaks at 10 GB + median peak , and res