Linux Shell: MOD returning null?

axipher · Feb 23, 2013

I have a rather odd problem here. I'm using the "%" operator to check if the current hour is a multiple of 3 for an update script. The script works for every hour except "9" where it simply returns null.

Here's the script:

Code:

#!/bin/bash

cd /media/mirror/folding/OCN/

NOWDATE=`date +%y-%m-%d_%H`
CURHOUR=`date +%H`

MOD_RESULT=$(( $CURHOUR % 3 ))

echo "-------------------------------------------------------------------------"

echo "TIMESTAMP:"

date

echo "MOD_RESULT: $MOD_RESULT"

if [ $MOD_RESULT == 0 ];
then
echo "3-hour update started"
ruby ./p_script.rb

python ./p_import.py
else
echo "Using previous update file"

python ./p_import.py
fi
echo "" >> ./p_log.txt

Here's the output from further on in the script:

Code:

---------------------------------------------------------------------------------------
TIMESTAMP:
Sat Feb 23 00:34:58 UTC 2013
MOD_RESULT: 0
3-hour update started
Auto Hourly Update for: 23-Feb-2013 @ 0:00 using file: passkey_output.csv
Adding new day: 23-Feb-2013
Updating user info for hour 0

---------------------------------------------------------------------------------------
TIMESTAMP:
Sat Feb 23 01:33:04 UTC 2013
MOD_RESULT: 1
Using previous update file
Auto Hourly Update for: 23-Feb-2013 @ 1:00 using file: passkey_output.csv
Updating user info for hour 1

---------------------------------------------------------------------------------------
TIMESTAMP:
Sat Feb 23 02:31:45 UTC 2013
MOD_RESULT: 2
Using previous update file
Auto Hourly Update for: 23-Feb-2013 @ 2:00 using file: passkey_output.csv
Updating user info for hour 2

---------------------------------------------------------------------------------------
TIMESTAMP:
Sat Feb 23 03:32:57 UTC 2013
MOD_RESULT: 0
3-hour update started
Auto Hourly Update for: 23-Feb-2013 @ 3:00 using file: passkey_output.csv
Updating user info for hour 3

---------------------------------------------------------------------------------------
TIMESTAMP:
Sat Feb 23 04:32:46 UTC 2013
MOD_RESULT: 1
Using previous update file
Auto Hourly Update for: 23-Feb-2013 @ 4:00 using file: passkey_output.csv
Updating user info for hour 4

---------------------------------------------------------------------------------------
TIMESTAMP:
Sat Feb 23 05:33:06 UTC 2013
MOD_RESULT: 2
Using previous update file
Auto Hourly Update for: 23-Feb-2013 @ 5:00 using file: passkey_output.csv
Updating user info for hour 5

---------------------------------------------------------------------------------------
TIMESTAMP:
Sat Feb 23 06:33:00 UTC 2013
MOD_RESULT: 0
3-hour update started
Auto Hourly Update for: 23-Feb-2013 @ 6:00 using file: passkey_output.csv
Updating user info for hour 6

---------------------------------------------------------------------------------------
TIMESTAMP:
Sat Feb 23 07:33:18 UTC 2013
MOD_RESULT: 1
Using previous update file
Auto Hourly Update for: 23-Feb-2013 @ 7:00 using file: passkey_output.csv
Updating user info for hour 7

---------------------------------------------------------------------------------------
TIMESTAMP:
Sat Feb 23 08:32:53 UTC 2013
MOD_RESULT:
Using previous update file
Auto Hourly Update for: 23-Feb-2013 @ 8:00 using file: passkey_output.csv
Updating user info for hour 8

---------------------------------------------------------------------------------------
TIMESTAMP:
Sat Feb 23 09:33:08 UTC 2013
MOD_RESULT:
Using previous update file
Auto Hourly Update for: 23-Feb-2013 @ 9:00 using file: passkey_output.csv
Updating user info for hour 9

---------------------------------------------------------------------------------------
TIMESTAMP:
Sat Feb 23 10:33:01 UTC 2013
MOD_RESULT: 1
Using previous update file
Auto Hourly Update for: 23-Feb-2013 @ 10:00 using file: passkey_output.csv
Updating user info for hour 10

---------------------------------------------------------------------------------------
TIMESTAMP:
Sat Feb 23 11:33:09 UTC 2013
MOD_RESULT: 2
Using previous update file
Auto Hourly Update for: 23-Feb-2013 @ 11:00 using file: passkey_output.csv
Updating user info for hour 11

---------------------------------------------------------------------------------------
TIMESTAMP:
Sat Feb 23 12:33:37 UTC 2013
MOD_RESULT: 0
3-hour update started
Auto Hourly Update for: 23-Feb-2013 @ 12:00 using file: passkey_output.csv
Updating user info for hour 12

---------------------------------------------------------------------------------------
TIMESTAMP:
Sat Feb 23 13:33:32 UTC 2013
MOD_RESULT: 1
Using previous update file
Auto Hourly Update for: 23-Feb-2013 @ 13:00 using file: passkey_output.csv
Updating user info for hour 13

---------------------------------------------------------------------------------------
TIMESTAMP:
Sat Feb 23 14:35:46 UTC 2013
MOD_RESULT: 2
Using previous update file
Auto Hourly Update for: 23-Feb-2013 @ 14:00 using file: passkey_output.csv
Updating user info for hour 14

hajile · Feb 24, 2013

Why not just use cron to run the task?

axipher · Feb 24, 2013

Quote:

Originally Posted by hajile

Why not just use cron to run the task?

This is all part of a massive cron job, I have update_all.sh which calls u_update.sh, p_update.sh, h_update.sh, update_graphs.sh, and finally mysql_copy.sh. That script runs every hour and is designed to be run in that order with all the database stuff.

Still can figure out why form within the script, "$CURHOUR % 3" returns 0 when $CURHOUR = 9. Today when I pulled up the log, I noticed it also returned null today (the 24th) at the 8 AM update.

Plan9 · Feb 25, 2013

According to your output, hour 8 is failing as well:
Quote:

TIMESTAMP:
Sat Feb 23 08:32:53 UTC 2013
MOD_RESULT: [blank]
Using previous update file
Auto Hourly Update for: 23-Feb-2013 @ 8:00 using file: passkey_output.csv
Updating user info for hour 8

It's also not your on show:

Code:

Code:

$ date +%H
14
$ echo $((9 % 3))
0
$ echo $((8 % 3))
2

So there's something else breaking your script that isn't visible from the code you posted. (and I'm guessing there is more code as there's output in your post that doesn't correlate with any echo's in your code (namely the "Updating user info for hour n" line)

axipher · Feb 25, 2013

Here is the entire script "p_update.sh"

Code:

Code:

#!/bin/bash

cd /media/mirror/folding/OCN/

NOWDATE=`date +%y-%m-%d_%H`

CURHOUR=`date +%H`

MOD_RESULT=$(( $CURHOUR % 3 ))

URL1="https://docs.google.com/spreadsheet/pub?key=....."

FILENAME1="./TC.csv"

FILENAME5="passkey_input.csv"

FILENAME6="passkey_output.csv"

echo "---------------------------------------------------------------------------------------" >> ./p_log.txt

echo "TIMESTAMP:" >> ./p_log.txt

date >> ./p_log.txt

echo "CURHOUR: $CURHOUR" >> ./p_log.txt

echo "MOD_RESULT: $MOD_RESULT" >> ./p_log.txt

if [ $MOD_RESULT == 0 ];

then

   echo "3-hour update started" >> ./p_log.txt

   wget -O $FILENAME1 $URL1 >/dev/null 2>&1

   cat $FILENAME1 > $FILENAME5

   echo >> $FILENAME5

   rm $FILENAME1

   ruby ./p_script.rb > $FILENAME6

   rm $FILENAME5

   python ./p_import.py $FILENAME6 >> ./p_log.txt

   cp ./$FILENAME6 ./archive/passkey/$NOWDATE

else

   echo "Using previous update file" >> ./p_log.txt

   python ./p_import.py $FILENAME6 >> ./p_log.txt

fi

echo "" >> ./p_log.txt

Plan9 · Feb 25, 2013

Your code looks good from what I can tell. Is that output you posted in the OP definitely verbatim?

What's your crontab like? Is there any chance that you have 8am and 9am calling an earlier version of the script by accident? or are you just calling the same script from the same crontab entry (ie a wildcard hour).

axipher · Feb 25, 2013

Quote:

Originally Posted by Plan9

Your code looks good from what I can tell. Is that output you posted in the OP definitely verbatim?

What's your crontab like? Is there any chance that you have 8am and 9am calling an earlier version of the script by accident? or are you just calling the same script from the same crontab entry (ie a wildcard hour).

That is the exact output in p_log.txt and I have a wildcard cron job that executes at half past the hour, every hour.

Plan9 · Feb 25, 2013

Quote:

Originally Posted by axipher

That is the exact output in p_log.txt and I have a wildcard cron job that executes at half past the hour, every hour.

Weird as there's lines in your code that doesn't appear in the output. eg in the code you have:

Code:

Code:

echo "TIMESTAMP:" >> ./p_log.txt

date >> ./p_log.txt

echo "CURHOUR: $CURHOUR" >> ./p_log.txt

echo "MOD_RESULT: $MOD_RESULT" >> ./p_log.txt

which I'd expect to see something like:

Code:

Code:

TIMESTAMP:
Mon Feb 25 16:42:40 GMT 2013
CURHOUR: 16
MOD_RESULT: 1

But there's no sign of "CURHOUR" in your output. It's not even as if you've typoed the output file names or done something else similar, the syntax is perfect, yet there's no sign of that line in the output

tom.slick · Feb 25, 2013

09: value too great for base (error token is "09")
http://ubuntuforums.org/showthread.php?t=677751

Numerical values starting with a zero (0) are interpreted as numbers in octal notation by the C language. As the only digits allowed in octal are {0..7}, an 8 or a 9 will cause the evaluation to fail.

ANSWER
You can explicitly state the base of a number using base#number

Code:

Code:

MOD_RESULT=$(( 10#$CURHOUR % 3 ))

Plan9 · Feb 25, 2013

Quote:

Code:
Originally Posted by tom.slick

09: value too great for base (error token is "09")
http://ubuntuforums.org/showthread.php?t=677751

Numerical values starting with a zero (0) are interpreted as numbers in octal notation by the C language. As the only digits allowed in octal are {0..7}, an 8 or a 9 will cause the evaluation to fail.

ANSWER
You can explicitly state the base of a number using base#number

Code:

Code:

MOD_RESULT=$(( 10#$CURHOUR % 3 ))

Nice catch. I would never have found that.

axipher · Feb 25, 2013

Quote:

Code:
Originally Posted by tom.slick

09: value too great for base (error token is "09")
http://ubuntuforums.org/showthread.php?t=677751

Numerical values starting with a zero (0) are interpreted as numbers in octal notation by the C language. As the only digits allowed in octal are {0..7}, an 8 or a 9 will cause the evaluation to fail.

ANSWER
You can explicitly state the base of a number using base#number

Code:

Code:

MOD_RESULT=$(( 10#$CURHOUR % 3 ))

Wow, thanks a bunch, made the change, we will see what happens tomorrow morning.

frozne · Feb 25, 2013

Nice, that one had me curious. If it is only considered octal when starting with 0, you can also use the date command and pass in a hyphen so that the hour field isn't padded with a 0.

Plan9 · Feb 25, 2013

Quote:

Originally Posted by frozne

Nice, that one had me curious. If it is only considered octal when starting with 0, you can also use the date command and pass in a hyphen so that the hour field isn't padded with a 0.

It seems like that's the case (re zero padding).

When I did my tests, I wasn't padding the numbers, which is why I missed that issue. Though even if I had spotted it, I still wouldn't have realised why it failed. It's a handy thing to know

tom.slick · Feb 26, 2013

The only reason I found it was because ran this to see the output and it gave me an error, also it happened to be just before 10am (9:57am).
Minutes later no more error. Just lucky to read this post at that time.

Code:

Code:

[email protected]:~/Documents> ./tt
./tt: line 7: 09: value too great for base (error token is "09")
-------------------------------------------------------------------------
TIMESTAMP:
Mon Feb 25 09:28:12 PST 2013
CURHOUR: 09
MOD_RESULT: 
[email protected]:~/Documents>

#!/bin/bash

NOWDATE=`date +%y-%m-%d_%H`
CURHOUR=`date +%H`

MOD_RESULT=$(( $CURHOUR % 3 ))

echo "-------------------------------------------------------------------------"

echo "TIMESTAMP:"

date
echo "CURHOUR: $CURHOUR"
echo "MOD_RESULT: $MOD_RESULT"

axipher · Feb 26, 2013

Definitely a handy trick to know and will come in very handy for a few other backup scripts I'm working on with similar errors based off the 0 padding issue.

Linux Shell: MOD returning null?

axipher

Moderator

hajile

Premium Member

axipher

Moderator

Plan9

Premium Member

axipher

Moderator

Plan9

Premium Member

axipher

Moderator

Plan9

Premium Member

tom.slick

Premium Member

Plan9

Premium Member

axipher

Moderator

frozne

Registered

Plan9

Premium Member

tom.slick

Premium Member

axipher

Moderator