Request() to api sometimes causes error: work-horse terminated unexpectedly

Redis client libraries (Java, Python, JS, etc.)
1

Hello,

I hope this is the right place to ask a programming question. I tried github, but am still waiting for a reply two days out.

I’m using python (3.9) on ubuntu locally and on the heroku server. I’m using their redis and rq to run a worker to run a function on a separate process. Well, the function runs for a while, but after about 20 to 30 requests() to an api (countdown, which is for ebay), the process terminates and I get the following error:
"Killed horse pid 10
2022-10-27T20:22:31.356467+00:00 app[worker.1]: 20:22:31 Moving job to FailedJobRegistry (work-horse terminated unexpectedly; waitpid returned None)"

The “request” function succeeds most of the time. However, when the child process fails it always fails at that point in the program.

Note, the program works without a problem when I don’t use redis and even when I spawn a child process without redis and run the function there. I have to use redis for heroku, however.

Thanks for any help!

Regards,
…John

2

It’s pretty hard to get from your description what error do you get, can you pleas share more details? which redis client do you use? do you have error log?

3

Hello! Thanks for the reply!

To give you an idea, I’m essentially using the code setup recommended by heroku here: Heroku rq

worker.work() runs the loop for checking for jobs and I enqueue jobs in my main python program.

It’s a lot of code to supply, but I run a function by queuing it up. The function does a lot of requests to the countdown api (ebay). It works smoothly most of the time, but every 25 or so requests to countdown causes the function/task to fail and throws the error shown above.

Regarding client, by client I assume you mean which machine? On my local machine the client is my machine and the redis server is on my machine. The python program on ubuntu. On heroku, where I want to run it ultimately, it’s the python program on the heroku server.

The error I get is what I posted above. I got that from my log file. I’ll repost below:

"Killed horse pid 10
2022-10-27T20:22:31.356467+00:00 app[worker.1]: 20:22:31 Moving job to FailedJobRegistry (work-horse terminated unexpectedly; waitpid returned None)"

I am new to redis, so there is probably something rudimentary I’m missing.

Let me know if that’s clear.

Thanks,
…John