Adding jobs to queue from code

[pabloriera]

Hi! In the docs says:

You can queue jobs from your Python code to avoid using mrq-run from the command-line.

But in the crawling example it uses the command-line.

How could I add jobs to a queue without the command-line?

Thaks

[sylvinus]

Hi!

Actually the crawler example does queue a job from its code:
https://github.com/pricingassistant/mrq/blob/master/examples/simple_crawler/crawler.py#L61

Is that what you were looking for?

[pabloriera]

Ok, yes, silly me. I thought there was another way. The thing is that I am having this error when doing a queue_jobs call

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-2-4e9de16fc7f8> in <module>()
      1 params={'url':'http://docs.tweepy.org'}
----> 2 queue_job('mrq_test.Fetch',params)

/usr/local/lib/python3.5/dist-packages/mrq/job.py in queue_job(main_task_path, params, **kwargs)
    609     """ Queue one job on a regular queue """
    610 
--> 611     return queue_jobs(main_task_path, [params], **kwargs)[0]
    612 
    613 

/usr/local/lib/python3.5/dist-packages/mrq/job.py in queue_jobs(main_task_path, params_list, queue, batch_size)
    623 
    624     from .queue import Queue
--> 625     queue_obj = Queue(queue, add_to_known_queues=True)
    626 
    627     if queue_obj.is_raw:

/usr/local/lib/python3.5/dist-packages/mrq/queue.py in __init__(self, queue_id, add_to_known_queues)
     73             self.root_id = self.id.split(delimiter)[0]
     74 
---> 75         self.use_large_ids = context.get_current_config()["use_large_job_ids"]
     76 
     77         # If this is the first time this process sees this queue, try to add it

KeyError: 'use_large_job_ids'

[fernandocamargoai]

I have the same problem.

[fernandocamargoai]

The following code worked for me:

from mrq import job, context

context.setup_context()

job.queue_job("tasks.exp1.ReqTask", {}, queue="gpu-based")

[sylvinus]

OK, thanks for your reports!

We're going to add a warning when using get_current_config() (so it will be triggered accross MRQ) when setup_context() wasn't called previously.

[mongin]

Warning added in 68f5240.