How does one check whether a task is running in celery (specifically, I"m using celery-django)?
I"ve read the documentation, and I"ve googled, but I can"t see a call like:
my_example_task.state() == RUNNING
My use-case is that I have an external (java) service for transcoding. When I send a document to be transcoded, I want to check if the task that runs that service is running, and if not, to (re)start it.
I"m using the current stable versions - 2.4, I believe.
AsyncResult object from the task id is the way recommended in the FAQ to obtain the task status when the only thing you have is the task id.
However, as of Celery 3.x, there are significant caveats that could bite people if they do not pay attention to them. It really depends on the specific use-case scenario.
In order for Celery to record that a task is running, you must set
True. Here is a simple task that tests this:
@app.task(bind=True) def test(self): print self.AsyncResult(self.request.id).state
False, which is the default, the state show is
PENDING even though the task has started. If you set
True, then the state will be
PENDINGmeans "I don"t know."
AsyncResult with the state
PENDING does not mean anything more than that Celery does not know the status of the task. This could be because of any number of reasons.
For one thing,
AsyncResult can be constructed with invalid task ids. Such "tasks" will be deemed pending by Celery:
>>> task.AsyncResult("invalid").status "PENDING"
Ok, so nobody is going to feed obviously invalid ids to
AsyncResult. Fair enough, but it also has for effect that
AsyncResult will also consider a task that has successfully run but that Celery has forgotten as being
PENDING. Again, in some use-case scenarios this can be a problem. Part of the issue hinges on how Celery is configured to keep the results of tasks, because it depends on the availability of the "tombstones" in the results backend. ("Tombstones" is the term use in the Celery documentation for the data chunks that record how the task ended.) Using
AsyncResult won"t work at all if
True. A more vexing problem is that Celery expires the tombstones by default. The
result_expires setting by default is set to 24 hours. So if you launch a task, and record the id in long-term storage, and more 24 hours later, you create an
AsyncResult with it, the status will be
All "real tasks" start in the
PENDING state. So getting
PENDING on a task could mean that the task was requested but never progressed further than this (for whatever reason). Or it could mean the task ran but Celery forgot its state.
AsyncResultwon"t work for me. What else can I do?
I prefer to keep track of goals than keep track of the tasks themselves. I do keep some task information but it is really secondary to keeping track of the goals. The goals are stored in storage independent from Celery. When a request needs to perform a computation depends on some goal having been achieved, it checks whether the goal has already been achieved, if yes, then it uses this cached goal, otherwise it starts the task that will effect the goal, and sends to the client that made the HTTP request a response that indicates it should wait for a result.
It would be easy for me to develop native apps using Java, C++ or Objective-C and I am also able to learn Kotlin, Dart or Swift, but things are much easier when you just use Python. I have done a Djan...
Python: - The Bible - 3 Manuscripts in 1 book:
It’s in all of us. Data science is what makes us humans what we are today. No, not the computer-driven da...
Data and storage models are the basis for big data ecosystem stacks. While storage model captures the physical aspects and features for data storage, data model captures the logical representation and...