65

My python script is not running under my crontab.

I have placed this in the python script at the top: 

#!/usr/bin/python

I have tried doing this:

chmod a+x myscript.py

Added to my crontab -e:

SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=""

* * * * * /home/me/project/myscript.py

My /var/log/cron file says:

Sep 21 11:53:02 163-dhcp /USR/SBIN/CROND[2489]: (me) CMD (/home/me/project/myscript.py)

But my script is not running because when I check my sql database, nothing has changed. If I run it directly in the terminal like so:

python /home/me/project/myscript.py

I get the correct result.

This is the myscript.py:

#!/usr/bin/python

import sqlite3

def main():
    con = sqlite3.connect("test.db")

    with con:

        cur = con.cursor()

        cur.execute("CREATE TABLE IF NOT EXISTS testtable(Id INTEGER PRIMARY KEY, Name TEXT)")

        cur.execute("INSERT INTO testtable(Name) VALUES ('BoB')")

        cur.execute("SELECT * FROM testtable")

        print cur.fetchall()    

if __name__ == "__main__":
    main()

Per comments: Yes, /usr/bin/python exists. I can also run the python script directly using just /home/me/project/myscript.py. /usr/bin/python /home/me/project/myscript.py works. So I don't believe this is the cause?

Improve this question
7
  • whats in the script? Does it depend on anything environmental? Commented Sep 21, 2012 at 16:08
  • Yes, I think we need to see inside your myscript.py Commented Sep 21, 2012 at 16:09
  • 2
    Set the MAILTO variable to your username. Then you will get some error message in your mail which might provide a clue. Commented Sep 21, 2012 at 16:11
  • 3
    You could also verify that running it directly with /home/me/project/myscript.py works (without the python at the start, which is how cron is running it). Commented Sep 21, 2012 at 16:12
  • 1
    Is test.db in your home directory or somewhere else? cron jobs get run from your home directory, so you may need to either change directories, or use a full path name to locate the database. Commented Sep 21, 2012 at 16:36

15 Answers 15

Reset to default
70

There are a lot of half answers across the internet so I thought I would capture this to save someone else some time.

First, cronjob does a poor job of telling you where this is failing. I recommend sending stderr output to a log file like this:

Crontab Command:

# m h  dom mon dow   command
* * * * * /path/to/your_file.sh >> out.txt  2>&1

As this is likely running the command as user, check home directory for the log file. Note this script runs every minute which is good for debugging.

The next issue is you probably have a path problem... as script likely is trying to execute from your home directory. This script sets the current directory, echos it to file, and then runs your program.

Try this :

Script File

#!/bin/sh
cd "$(dirname "$0")";
CWD="$(pwd)"
echo $CWD
python your_python_file.py

Hope this saves someone else some debugging time!!!

Improve this answer
Sign up to request clarification or add additional context in comments.

6 Comments

Image
What does the last part "2>&1" means?
Image
I have the same question as @Nechadil what does the 2>&1 does. Thanks in advance!
Image
@EnriqueBet I've found the answer afterwards. check this link :) unix.stackexchange.com/questions/99263/…
Image
Hi, @TiagoLubiana do you know what is meant by script file? Where do i have to type this.
Image
2>&1 means you're redirecting the error output (2) to the same location as regular output (&1). If you run a script, normally they both end up on the screen, but if you're trying to save the output to a file you have to tell it to send both types of output to the file
|
45

What happens when you type

/home/me/project/myscript.py into the shell?

Can you explicitly use /usr/bin/python in your crontbb command?

Can you either use an absolute path to your test.db or cd to the correct directory then execute your python script?

This is helpful to have debug statements in your python and log some data. Crontab can be very tricky to debug.

Improve this answer

2 Comments

Image
Haha, you did it! I changed the test.db to use an absolute path /home/me/project/test.db... And it works!
Image
The cron job runs in /home/me and apparently you were imagining that it would run in /home/me/project. You don't need absolute paths as long as you understand (relative paths and) where cron starts jobs.
20

It's usually because the python used by crontab is different from the one you use in the shell. The easiest way to solve this is:

  1. get the python you use in the shell:

$ which python # it may be "python3" or something else /usr/bin/python

  1. use that specific python in crontab file:

* * * * * /usr/bin/python test.py

Also want to mention that using env -i /bin/bash --noprofile --norc in the shell lets you have the same environment as the one used by crontab, and this is super helpful to debug.

Improve this answer

Comments

19

It is possible that the script does not start because it cannot locate the python interpreter. Crontab environment may be very different from the shell environment which you are using. The search paths may be differ significantly. Also, you test your script by starting the python interpreter explicitly while you expect the crontab to only start the script. I put this line at the top of my python scripts:

\#!/bin/env python

This line will help locate the interpreter regardless of which directory it is installed in as long as it is in the search path.

Improve this answer

5 Comments

Image
I am currently in bash, and I've added your suggestion #!/bin/env python. It didn't work it seems.
Image
First, see if you have /bin/env or if env is in a different directory.
Image
Start the following command to have the same environment as the one used by crontab: env -i /bin/bash --noprofile --norc Then start your python script and see why it fails. This should give you an idea.
Image
@shargors I can't tell you how much your comment helped me. In my opinion, your comment should be the accepted answer: the particulars of why the python script failed in the OP's case aren't as important as the general way to solve environment problems. Thanks! :)
Image
@shargors You may want to add your comment to the answer. Even though it's not the "answer" to the OPs question, it seems like it has helped A LOT of people (me included).
10

Typically, crontab problems like this are caused by the PATH environment variable being more restrictive/different than what your normal user's PATH environment is. Since your shell uses the PATH environment to find the executable (e.g. /usr/bin/python is found in /usr/bin when you type "python" at a shell prompt), when the PATH is missing common locations, like /usr/bin or /usr/sbin, your cron job will fail. This has bit me many times. The simple fix is just to explicitly set the PATH yourself near the top of your crontab file, before any commands that need it. So, just edit the crontab as usual and add something like this near the top (if your binary is not in one of the below paths, you'll need to add it after a colon):

PATH=/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin

Alternately, just use absolute paths to your binaries and scripts in crontab.

Improve this answer

Comments

5

Try this 

* * * * * cd <directory_where_python_file_is> && bin/app etc/app_defaults.yaml

There is some path issue with cron. So when you move to directory with python file, cron works like charm!

Improve this answer

Comments

3

I'd got the same problem. Despite the fact that the script executed manually was working, in crontab no options mentioned above were working at all. I've moved my script from /home/user/script_directory/ to /opt/scripts/ and it started to work. Possible cause of the problem should be the access (read) permissions to subfolder located in home directory.

Improve this answer

Comments

3

Easiest way to handle this is to add your python installation's path to PATH in top of the shell script. Something like:

#!/usr/bin/env bash
export PATH="{path to your python installation}:$PATH"
python {python_file_name}.py

As @Shargors said you can test it by 

env -i /bin/bash --noprofile --norc
Improve this answer

Comments

3

If you are using anaconda for python then the path to use will be :

/home/username/anaconda3/bin/python test.py
Improve this answer

Comments

1

While the answers here clearly delineate the problem and solution, I wanted to add another answer which helped me.

If your python script is calling a database, then be sure you can connect to the db properly within the cron env (to identify the cron env--> https://askubuntu.com/questions/23009/reasons-why-crontab-does-not-work). I had a file that would run from the shell, but not as a crontab unless I connected to the database as root from within the python script.

Improve this answer

Comments

1

Sometimes I am facing same problem. Whatever I try something as advised here, I may not get result.

So I begin to write "trigger" bash script as follow (let's name it trigger.sh):

#!/bin/bash

/full_path/python_script.py

And I am calling trigger.sh from crontab and everything is fine.

EDIT: Of course, don't forget to do following (give execution right):

$chmod +x python_script.py
$chmod +x trigger.sh
Improve this answer

Comments

1

I was working on project that includes paramiko lib, when I run the Check_.py from cmdlin it works perfect but when I set the crontab it fails with error no module name paramiko.

So to make it short:
- there were two different python versions installed 3.7 and 2.4, so I used whreris python3 to locate the python path /usr/local/bin/python3.7m so replacing the python with the path will solve the issue.

Example

* * * * * cd /home/MKhair/hlthchk/BR/ && /usr/local/bin/python3.7m /home/MKhair/hlthchk/BR/Check_.py
* * * * * cd [ path-to-the-script-dir] && [path-to-python] [path-to-the-script]
Improve this answer

Comments

0

Try to put in your crontab:

* * * * * python /path/to/your/script.py

rather than

* * * * * /path/to/your/script.py

Also the shebang line is #!/usr/bin/env python in some environments. env is an executable, and you have to know where it lives with "$ which env".

Improve this answer

Comments

0
  • Is the cron user (where the script fails) and the terminal user (when the script succeeds) are same ?
  • Can you redirect the job output to some file as mentioned in Cron Job Log - How to Log?. We could see whether that helps.
Improve this answer

Comments

0

This might be helpful for someone. I was having this same issue (or at least a similar issue) and what helped me was to get the path in which Python (Be aware of the version you want to use python, python3, etc...) by running this:

which python3

And then, I replaced python3 for the full path of python3 in my crontab file.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.