docker-lambda

bdas (128MB tier) are terrible at cold starts if you have lots of stuff to import (which WILL BE THE CASE for Zappa applications!). If a cold start exceeds your timeout (which defaults to 30 sec), your Lambda will never complete and never get out of cold start. 6. If you're playing with S3, PUTs will hurt your wallet. A common application of Lambda (the famed "resize images" example) has a workflow that looks like this: GET on API gateway, PUT to s3, triggers an event which fires a Lambda, lambda processes payload, PUTs result back to S3. If processing your payload is fast, the PUTs in this case could be the most expensive part of your flow. 7. X-Ray is atrociously confusing. 8. No cloudwatch metrics for memory usage. Had to do my own using log metrics. Dumb. 9. There's a cool versioning system for the code artifact, kinda like ECS has. It's really nice to use except that unlike ECS it doesn't have native lifecycle rules, which means you have to do your own cleanups. Dumb. 10. Lambda patterns can be implemented more cheaply and more efficiently using tasks queues and autoscaling EC2s. Amazon's "efficient" compute allocation is outweighed by the margins on Lambda pricing. As with other Amazon services, you're paying for the system to be managed. 11. Aurora Serverless is super interesting, but FWICT nowhere near ready for prime time. 12. API gateway is a solution looking for a problem. If you have to use it (eg. you must trigger your lambda through an http request and you don't want to run a web server), use it in proxy mode. 13. Golang lambdas are very promising. I want to try them out. 14. There's a native canary system in Lambda/APIGW nowadays. Also want to try it out. Had to roll my own before it existed and now too scared to touch the production system. 15. If you hit concurrency issues, your problem might not be load, it might be throughput. Is your DB slower than usual? Common pattern: Increase in Lambda requests hitting your DB, slows your DB down, which in turn slows your lambdas down, which causes you to hit concurrency limits. All these metrics are tightly related. 16. Your tests should not assume a Lambda environment unless your app actually depends on the lambda environment (it generally shouldn't). 17. Use this thing: https://github.com/lambci/docker-lambda - Among other things, you can use it to build binary libs/dependencies you want to run on Lambda (cairo for example). 18. Use this thing: https://github.com/spulec/moto -- And also this thing: https://github.com/localstack/localstack 19. Cloudwatch logs suck. If you want your app to be debuggable, make sure you know how to find logs for individually-triggered lambdas (eg. if you're thumbnailing, you should be able to trace the logs for that one thumbnail back into cloudwatch easily) 20. Use Sentry. https://sentry.io/ - You can use the Sentry extra context parameters to help auditing various things (lambda environment, aws api results you get at runtime, etc)

mulating lambda, and the "lambda:build" container makes building things easy and repeatable: https://github.com/lambci/docker-lambda. In a pinch, you can build on an Amazon Linux EC2 instance and it should work on Lambda, but you will have to be more careful about dynamic linking. As another option: I'm not sure if it's been mentioned, but you can also try a ready-made OCR service before packaging up Tesseract, like this one: https://algorithmia.com/algorithms/ocr/SmartOCR. So anyway, the performance part has good solutions, at least. For fixing the accuracy: I know next to nothing about approximate string matching, but perhaps it would then be possible to do a fuzzy search over the text using something similar a Levenshtein automaton: https://en.wikipedia.org/wiki/Levenshtein_automaton. You may also want to take a look at this: https://en.wikipedia.org/wiki/Bag-of-words_model More broadly, I'm sure that there are text-based document classification methods that are robust against sloppy OCR. It may just take some research on the main approaches people take to document classification -- it's not my area, but my understanding is that this is typically approached with statistical methods. Otherwise your spam filter would get defeated by typos.