后台工作：从定时任务到分布式系统
本文探讨后台工作的必要性，从单机定时脚本到分布式系统的演进，并介绍多种后台工作策略。
中文处理结果
考虑当有人向 Web 应用上传个人资料照片时会发生什么。
根据用例，应用可能会将图像调整为不同尺寸。它可能会对图像运行内容检查流程。它还可以将调整后的图像推送到内容分发网络以加快访问速度。最后，它还会用最新的元数据更新用户记录。现在，想象一下所有这些操作都在同一次图片上传请求路径中发生。当所有这些功能都执行时，用户会看到旋转按钮持续几秒钟，想知道照片是否上传成功。不用说，这将是一种相当糟糕的用户体验。
如果我们将所有这些额外工作移出请求路径，上传几乎可以立即响应，只要图像文件存储在对象存储中。图像仍然会被调整大小、扫描和分发，只是不在同一流程中。这被称为后台工作，以下是一些需要它的原因：
- 用户操作可能触发了它。例如，有人注册，然后发送欢迎邮件。
- 时钟可能触发了它。例如，夜间报告、月度发票、每小时缓存刷新。
- 另一个系统可能触发了它。例如，webhook 到达，或文件存储在对象存储中。
- 也许庞大的工作量使其值得。例如，某些工作自然更适合一次处理一千项，更便宜或更安全。
大多数情况下，团队从单台机器上的单个计划脚本开始。它可以处理大量后台工作。然而，随着系统变得更大更复杂，后台工作的数量需要不同的策略。在本文中，我们将详细研究执行后台工作的各种策略。
为什么普通请求不能在所有地方工作
阅读更多
原始正文摘录
Background Work: From Cron Jobs to Distributed Systems
Consider what can happen when someone uploads a profile photo to a web application.
Depending on the use case, the application may resize the image into different sizes. It might run the image through a content check process. It can also push the resized images to a content delivery network for faster access. Lastly, it will also update the user record with the latest metadata. Now, imagine if all of these operations take place during the same picture upload request path. As all of these functionalities are carried out, the user sees a spinning button for several seconds, wondering whether the photo has been uploaded successfully. Needless to say, it would be a pretty poor user experience.
If we move all of that extra work outside the request path, the upload can more or less respond immediately as soon as the image file is stored in an object storage. The image still gets resized, scanned, and distributed, just not in the same flow. This is known as background work, and here are some examples of why it is needed:
- A user action may have triggered it. For example, someone signs up, and a welcome email goes out.
- The clock may have triggered it. For example, nightly reports, monthly invoices, hourly cache refreshes.
- Another system could have triggered it. For example, a webhook arrives, or a file is stored in object storage.
- Perhaps the sheer volume of work made it worthwhile. For example, some work is naturally cheaper or safer done a thousand items at a time.
Most of the time, teams start with a single scheduled script on a single machine. It can handle a great amount of background work. However, as the system gets bigger and more complex, the amount of background work requires different strategies. In this article, we will look at various such strategies to perform background work in detail.
Why Normal Requests Don’t Work Everywhere
Read more