在 Amazon Elastic Beanstalk 工作节点环境中运行 Rails 后台作业或定期任务 (cron 作业)。无需自定义容器命令。
在 Amazon Elastic Beanstalk 工作节点环境中运行 Rails 后台作业或定期任务 (cron 作业)。无需自定义容器命令。
You have your Rails application deployed on the Amazon Elastic Beanstalk platform and now your application needs to offload work—like sending emails—into asynchronous background jobs. Or you want to perform jobs periodically similar to cron jobs. Then Active Elastic Job is the right gem. It provides an adapter for Rails' Active Job framework that allows your application to queue jobs as messages in an Amazon SQS queue. Elastic Beanstalk provides worker environments that automatically pull messages from the queue and transforms them into HTTP requests. This gem knows how to handle these requests. It comes with a Rack middleware that intercepts these requests and transforms them back into jobs which are subsequently executed.
Add this line to your application's Gemfile:
gem 'active_elastic_job'
Create an SQS queue:
Log into your Amazon Web Service Console and select SQS from the services menu.
Create a new queue. Select a name of choice but do not forget to use the same name in your Active Job class definition.
class YourJob /var/proxy/staging/nginx/conf.d/read_timeout.conf > /var/log/sqsd.log 2>&1 &
fi* *worker-health-url* is optional, but better to have than to not
* *ssl-enabled* is set to false as the default Elastic Beanstalk setup has the SSL ending at the load balancer and not the application.
* *daemonized* is set to false otherwise `sqsd` would stop once the queue was empty (this seems backwards from the sqsd README, but it works this way)
* *user-agent* is technically optional as the default value is `sqsd`, but there's potential to expand features based on this field
* Everything starting with the `>>` is optional unless you want output from the daemon logged
#### Potential Problems
`aws-sqsd` cannot coordinate resources with `sqsd` therefor it can't properly "load balance" tasks like normal. If this becomes a problem you can lower the number of concurrent workers and max messages to help keep resources in check.
## FAQ
A summary of frequently asked questions:
### What are the advantages in comparison to popular alternatives like Resque, Sidekiq or DelayedJob?
You decided to use Elastic Beanstalk because it facilitates deploying and operating your application. Active Elastic Job embraces this approach and keeps deployment and maintenance simple. To use Resque, Sidekiq or DelayedJob as a queuing backend, you would need to setup at least one extra EC2 instance that runs your queue application. This complicates deployment. Furthermore, you will need to monitor your queue and make sure that it is in a healthy state.
### Can I run Resque or DelayedJob in my web environment which already exists?
It [is](http://junkheap.net/blog/2013/05/20/elastic-beanstalk-post-deployment-scripts/) [possible](http://www.dannemanne.com/posts/post-deployment_script_on_elastic_beanstalk_restart_delayed_job) but not recommended. Your jobs will be executed on the same instance that is hosting your web server, which handles your users' HTTP requests. Therefore, the web server and the worker processes will fight for the same resources. This leads to slower responses of your application. But a fast response time is actually one of the main reasons to offload tasks into background jobs.
### Is there a possibility to prioritize certain jobs?
Amazon SQS does not support prioritization. In order to achieve faster processing of your jobs you can add more instances to the worker environment or create a separate queue with its own worker environment for your high-priority jobs.
### Can jobs be delayed?
You can schedule jobs not more than **15 minutes** into the future. See [the Amazon SQS API reference](http://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SendMessage.html). If you need to postpone the execution of a job further into the future, then consider the possibility of setting up a periodic task.
### Can I monitor and inspect failed jobs?
Amazon SQS provides [dead-letter queues](http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/SQSDeadLetterQueue.html). These queues can be used to isolate and sideline unsuccessful jobs.
### Is my internet-facing web environment protected against being spoofed into processing jobs?
The Rails application will treat requests presenting a user agent value `aws-sqsd/*`
as a request from the SQS daemo; therefore, it tries to un-marshal the request body back into a job object for further execution. This adds a potential attack vector since anyone can fabricate a request with this user agent and, therefore, might try to spoof the application into processing jobs or even malicious code. This gem takes several counter-measures to block the attack vector.
* The middleware that processes the requests from the SQS daemon is disabled per default. It has to be enabled deliberately by setting the environment variable **PROCESS_ACTIVE_ELASTIC_JOBS** to `true`, as instructed in the [Usage](#usage) section.
* Messages that represent the jobs are signed before they are enqueued. The signature is verified before the job is executed. This is the reason both environments-web and worker-need to have the same value for the environment variable **SECRET_KEY_BASE** (see the [Usage](#usage) section Step 7) since the secret key base will be used to generate and verify the signature.
* Only requests that originate from the same host (localhost) are considered to be requests from the SQS daemon. SQS daemons are installed in all instances running in a worker environment and will only send requests to the application running in the same instance.
Because of these safety measures it is possible to deploy the same codebase to both environments, which keeps the deployment simple and reduces complexity.
### Can jobs get lost?
Active Elastic Job will raise an error if a job has not been sent successfully to the SQS queue. It expects the queue to return an MD5 digest of the message contents, which it verifies for correctness. Amazon advertises SQS to be reliable and messages are stored redundantly. If a job is not executed successfully, the corresponding message become visible in the queue again. Depending on the queue's setting, the worker environment will pull the message again and an attempt will be made to execute the jobs again.
### What can be the reason if jobs are not executed?
Inspect the log files of your worker tier environment. It should contain entries for the requests that are performed
by the AWS SQS daemon. Look out for POST requests from user agents starting with `aws-sqsd/`. If the log does not
contain any, then make sure that there are messages enqueued in the SQS queue which is attached to your worker tier. You can do this from
your AWS console.
When you have found the requests, check their response codes which give a clue on why a job is not executed:
* status code `500`: something went wrong. The job might have raised an error.
* status code `403`: the request seems to originate from another host than `localhost` or the message which represents the job has not been verified successfully. Make sure that both environment, web and worker, use the same `SECRET_KEY_BASE`.
* status code `404` or `301`: the gem is not included in the bundle, or the `PROCESS_ACTIVE_ELASTIC_JOBS` is **not** set to `true` (see step 6) in the worker environment or the worker environment uses an outdated platform which uses the AWS SQS daemon version 1. Check the user agent again, if it lookes like this `aws-sqsd/1.*` then it uses the old version. This gem works only for daemons version 2 or newer.
## Bugs - Questions - Improvements
Whether you catch a bug, have a question or a suggestion for improvement, I sincerely appreciate any feedback. Please feel free to [create an issue](https://github.com/active-elastic-job/active-elastic-job/issues/new) and I will follow up as soon as possible.
## Contribute
Running the complete test suite requires to launch elastic beanstalk environments. Travis builds triggered by a pull request will launch the needed elastic beanstalk environments and subsequently run the complete test suite. You can run all specs that do not depend on running elasitic beanstalk environments by setting an environment variable:
```bash
EXCEPT_DEPLOYED=true bundle exec rspec specFeel free to issue a pull request, if this subset of specs passes.
We recommend to run the test suite in a controlled and predictable envrionment. If your development machine has Docker installed, then you can make use of the Dockerfile that comes with this package. Build an image and run tests in container of that image.
docker build -t active-elastic-job-dev .
docker run -e EXCEPT_DEPLOYED=true -v $(pwd):/usr/src/app active-elastic-job-dev bundle exec rspec spec暂无开放 Issues,或尚未同步最近议题。