Skip to content

Commit 05d3902

Browse files
sylvinussylvinus
committed
Formatting fixes to docs
1 parent 08ebf36 commit 05d3902

1 file changed

Lines changed: 79 additions & 66 deletions

File tree

‎docs/command-line.md‎

Lines changed: 79 additions & 66 deletions
Original file line numberDiff line numberDiff line change
@@ -3,88 +3,101 @@
33
## Configuration flags
44

55
All the command-line tools support a set of common configuration flags, defined in [config.py](https://github.com/pricingassistant/mrq/blob/master/mrq/config.py). Use --help with any of them to see the full list.
6-
- **General arguments**:these can be passed to either **mrq-worker**,**mrq-run**, or **mrq-dashboard**
7-
- `--trace_greenlets`:Collect stats about each greenlet execution time and switches.Defaults to **false**.
8-
- `--trace_memory`:Collect stats about memory for each task. Incompatible with --greenlets > 1. Defaults to **false**.
9-
- `--trace_io`:Collect stats about all I/O operations.Defaults to **true**.
10-
- `--print_mongodb`:Print all MongoDB requests.Defaults to **false**.
11-
- `--trace_memory_type`:Create a .png object graph in trace_memory_output_dir with a random object of this type.
12-
- `--trace_memory_output_dir`:Directory where to output .pngs with object graphs.Defaults to folder **memory_traces**.
13-
- `--profile`:Run profiling on the whole worker.Defaults to **false**.
14-
- `--mongodb_jobs, --mongodb`: MongoDB URI for the jobs, scheduled_jobs & workers database.Defaults to **mongodb://127.0.0.1:27017/mrq**.
15-
- `--mongodb_logs`:MongoDB URI for the logs database."0" will disable remote logs, "1" will use main MongoDB.Defaults to **1**
16-
- `--mongodb_logs_size`:If provided, sets the log collection to capped to that amount of bytes.
17-
- `--no_mongodb_ensure_indexes`:If provided, skip the creation of MongoDB indexes at worker startup.
18-
- `--redis`:Redis URI.Defaults to **redis://127.0.0.1:6379**.
19-
- `--redis_prefix`:Redis key prefix.Default to "mrq".
20-
- `--redis_max_connections`:Redis max connection pool size.Defaults to **1000**.
21-
- `--redis_timeout`:Redis connection pool timeout to wait for an available connection.Defaults to **30**.
22-
- `--name`:Specify a different name.
23-
- `--quiet`:Don\'t output task logs.Defaults to **false**.
24-
- `--config, -c`:Path of a config file.
25-
- `--worker_class`:Path to a custom worker class.Defaults to **"mrq.worker.Worker"**.
26-
- `--version, -v`:Prints current MRQ version.Defaults to **false**.
27-
- `--no_import_patch`:Skips patching __import__ to fix gevent bug #108.Defaults to **false**.
28-
- `--add_network_latency`:Adds random latency to the network calls, zero to N seconds. Can be a range (1-2)').Defaults to **0**.
29-
- `--default_job_result_ttl`:Seconds the results are kept in MongoDB when status is success.Defaults to **604800** seconds which is 7 days.
30-
- `--default_job_abort_ttl`:Seconds the tasks are kept in MongoDB when status is abort.Defaults to **86400** seconds which is 1 day.
31-
- `--default_job_cancel_ttl`:Seconds the tasks are kept in MongoDB when status is cancelDefaults to **86400** seconds which is 1 day.
32-
- `--default_job_timeout`:In seconds, delay before interrupting the job.Defaults to **3600** seconds which is 1 hour.
33-
- `--default_job_max_retries`:Set the status to "maxretries" after retrying that many times.Defaults to **3** seconds.
34-
- `--default_job_retry_delay`:Seconds before a job in retry status is requeued again.Defaults to **3** seconds.
35-
- `--use_large_job_ids`:Do not use compacted job IDs in Redis. For compatibility with 0.1.x only. Defaults to **false**.
36-
- `mrq-worker` starts a worker
37-
- `queues`: The queues to listen on.Defaults to **default** , which will listen on all queues.
38-
- `--max_jobs`:Gevent:max number of jobs to do before quitting. Temp workaround for memory leaks.Defaults to **0**
39-
- `--max_memory`:Max memory (in Mb) after which the process will be shut down. Use with `--processes [1-N]`
40-
to have supervisord automatically respawn the worker when this happens.Defaults to **1**
41-
- `--grenlets, --gevent, --g`:Max number of greenlets to use.Defaults to **1**.
42-
- `--processes, --p`:Number of processes to launch with supervisord.Defaults to **0**.
43-
- `--supervisord_template`:Path of supervisord template to use. Defaults to **supervisord_templates/default.conf**.
44-
- `--scheduler`:Run the scheduler.Defaults to **false**.
45-
- `--scheduler_interval`:Seconds between scheduler checks.Defaults to **60** seconds, only ints are acceptable.
46-
- `--report_interval`:Seconds between worker reports to MongoDB.Defaults to **10** seconds, floats are acceptable too.
47-
- `--report_file`:Filepath of a json dump of the worker status. Disabled if none.
48-
- `--admin_port`:Start an admin server on this port, if provided. Incompatible with --processes.Defaults to **0**
49-
- `--admin_ip`:IP for the admin server to listen on. Use "0.0.0.0" to allow access from outside.Defaults to **127.0.0.1**.
50-
- `--local_ip`:Overwrite the local IP, to be displayed in the dashboard.
51-
- `--max_latency`:Max seconds while worker may sleep waiting for a new job.Can be < 1 and a float value.
52-
53-
- `mrq-dashboard` starts the web dashboard on the default port.
54-
- `--dashboard_httpauth`:HTTP Auth for the Dashboard. Format is user:pass.
55-
- `--dashboard_queue`:Default queue for dashboard actions.
56-
- `--dashboard_port`:Use this port for mrq-dashboard.Defaults to port **5555**.
57-
- `--dashboard_ip`:Bind the dashboard to this IP. Default is **0.0.0.0**, use **127.0.0.1** to restrict access.
58-
59-
- `mrq-run` runs a task. If you add the `--queue` option that will enqueue it to be later ran by a worker.
60-
- `--queue`:Queue the task on this queue instead of running it right away.
61-
- `taskpath`:Task to run.
62-
- `taskargs`:JSON-encoded arguments, or "key value" pairs.
63-
646

65-
Typical usage is:
66-
```
67-
$ mrq-run tasks.mylib.myfile.MyTask '{"param1": 1, "param2": True}'
68-
```
7+
The following general flags can be passed as command-line arguments to either **mrq-worker**,**mrq-run**, or **mrq-dashboard**:
8+
9+
- `--trace_greenlets`:Collect stats about each greenlet execution time and switches.Defaults to **false**.
10+
- `--trace_memory`:Collect stats about memory for each task. Incompatible with --greenlets > 1. Defaults to **false**.
11+
- `--trace_io`:Collect stats about all I/O operations.Defaults to **true**.
12+
- `--print_mongodb`:Print all MongoDB requests.Defaults to **false**.
13+
- `--trace_memory_type`:Create a .png object graph in trace_memory_output_dir with a random object of this type.
14+
- `--trace_memory_output_dir`:Directory where to output .pngs with object graphs.Defaults to folder **memory_traces**.
15+
- `--profile`:Run profiling on the whole worker.Defaults to **false**.
16+
- `--mongodb_jobs, --mongodb`: MongoDB URI for the jobs, scheduled_jobs & workers database.Defaults to **mongodb://127.0.0.1:27017/mrq**.
17+
- `--mongodb_logs`:MongoDB URI for the logs database."0" will disable remote logs, "1" will use main MongoDB.Defaults to **1**
18+
- `--mongodb_logs_size`:If provided, sets the log collection to capped to that amount of bytes.
19+
- `--no_mongodb_ensure_indexes`:If provided, skip the creation of MongoDB indexes at worker startup.
20+
- `--redis`:Redis URI.Defaults to **redis://127.0.0.1:6379**.
21+
- `--redis_prefix`:Redis key prefix.Default to "mrq".
22+
- `--redis_max_connections`:Redis max connection pool size.Defaults to **1000**.
23+
- `--redis_timeout`:Redis connection pool timeout to wait for an available connection.Defaults to **30**.
24+
- `--name`:Specify a different name.
25+
- `--quiet`:Don\'t output task logs.Defaults to **false**.
26+
- `--config, -c`:Path of a config file.
27+
- `--worker_class`:Path to a custom worker class.Defaults to **"mrq.worker.Worker"**.
28+
- `--version, -v`:Prints current MRQ version.Defaults to **false**.
29+
- `--no_import_patch`:Skips patching __import__ to fix gevent bug #108.Defaults to **false**.
30+
- `--add_network_latency`:Adds random latency to the network calls, zero to N seconds. Can be a range (1-2)').Defaults to **0**.
31+
- `--default_job_result_ttl`:Seconds the results are kept in MongoDB when status is success.Defaults to **604800** seconds which is 7 days.
32+
- `--default_job_abort_ttl`:Seconds the tasks are kept in MongoDB when status is abort.Defaults to **86400** seconds which is 1 day.
33+
- `--default_job_cancel_ttl`:Seconds the tasks are kept in MongoDB when status is cancelDefaults to **86400** seconds which is 1 day.
34+
- `--default_job_timeout`:In seconds, delay before interrupting the job.Defaults to **3600** seconds which is 1 hour.
35+
- `--default_job_max_retries`:Set the status to "maxretries" after retrying that many times.Defaults to **3** seconds.
36+
- `--default_job_retry_delay`:Seconds before a job in retry status is requeued again.Defaults to **3** seconds.
37+
- `--use_large_job_ids`:Do not use compacted job IDs in Redis. For compatibility with 0.1.x only. Defaults to **false**.
6938

7039
## mrq-worker
71-
### Concurrency
40+
41+
`mrq-worker` starts a new worker and takes one argument list:
42+
43+
- `queues`: The queues to listen on.Defaults to **default** , which will listen on all queues.
44+
45+
You can pass additional configuration flags:
46+
47+
- `--max_jobs`:Gevent:max number of jobs to do before quitting. Temp workaround for memory leaks.Defaults to **0**
48+
- `--max_memory`:Max memory (in Mb) after which the process will be shut down. Use with `--processes [1-N]`
49+
to have supervisord automatically respawn the worker when this happens.Defaults to **1**
50+
- `--grenlets, --gevent, --g`:Max number of greenlets to use.Defaults to **1**.
51+
- `--processes, --p`:Number of processes to launch with supervisord.Defaults to **0**.
52+
- `--supervisord_template`:Path of supervisord template to use. Defaults to **supervisord_templates/default.conf**.
53+
- `--scheduler`:Run the scheduler.Defaults to **false**.
54+
- `--scheduler_interval`:Seconds between scheduler checks.Defaults to **60** seconds, only ints are acceptable.
55+
- `--report_interval`:Seconds between worker reports to MongoDB.Defaults to **10** seconds, floats are acceptable too.
56+
- `--report_file`:Filepath of a json dump of the worker status. Disabled if none.
57+
- `--admin_port`:Start an admin server on this port, if provided. Incompatible with --processes.Defaults to **0**
58+
- `--admin_ip`:IP for the admin server to listen on. Use "0.0.0.0" to allow access from outside.Defaults to **127.0.0.1**.
59+
- `--local_ip`:Overwrite the local IP, to be displayed in the dashboard.
60+
- `--max_latency`:Max seconds while worker may sleep waiting for a new job.Can be < 1 and a float value.
61+
62+
### Worker concurrency
7263

7364
The default is to run tasks one at a time. You should obviously change this behaviour to use Gevent's full capabilities with something like:
7465

75-
`mrq-worker --processes 3 --greenlets 10`
66+
`mrq-worker --processes 3 --greenlets 10 queue-highpriority queue-default`
7667

7768
This will start 30 greenlets over 3 UNIX processes. Each of them will run 10 jobs at the same time.
7869

7970
As soon as you use the `--processes` option (even with `--processes=1`) then supervisord will be used to control the processes. It is quite useful to manage long-running instances.
8071

81-
8272
### Simulating network latency
8373

8474
Sometimes it is helpful in local development to simulate an environment with higher network latency.
8575

8676
To do this we added a ```--add_network_latency=0.1``` config option that will add (in this case) a random delay between 0 and 0.1 seconds to every network call.
8777

78+
## mrq-dashboard
79+
80+
`mrq-dashboard` starts the web dashboard on the default port and takes these arguments:
81+
82+
- `--dashboard_httpauth`:HTTP Auth for the Dashboard. Format is user:pass.
83+
- `--dashboard_queue`:Default queue for dashboard actions.
84+
- `--dashboard_port`:Use this port for mrq-dashboard.Defaults to port **5555**.
85+
- `--dashboard_ip`:Bind the dashboard to this IP. Default is **0.0.0.0**, use **127.0.0.1** to restrict access.
86+
8887
## mrq-run
8988

90-
## mrq-dashboard
89+
`mrq-run` runs a one-off task. If you add the `--queue` option that will enqueue it to be later ran by a worker.
90+
91+
- `taskpath`:Task to run.
92+
- `taskargs`:JSON-encoded arguments, or "key value" pairs.
93+
- `--queue`:Queue the task on this queue instead of running it right away.
94+
95+
Typical usage is:
96+
97+
```
98+
$ mrq-run tasks.mylib.myfile.MyTask '{"param1": 1, "param2": True}'
99+
100+
# Shorter syntax which casts all values as strings (equivalent to '{"param1": "1", "param2": "ok"}')
101+
$ mrq-run tasks.mylib.myfile.MyTask param1 1 param2 ok
102+
```
103+

0 commit comments

Comments
 (0)