dotted quad to decimal and back, in bash

bash snippet to convert IP address to decimal:

$ cat ip2d.sh
#!/bin/bash

IFS=.
set -- $*
echo $(( ($1*256**3) + ($2*256**2) + ($3*256) + ($4) ))

Conversely, to convert from decimal to dotted quad address:

$ cat d2ip.sh
#!/bin/bash

IFS=" " read -r a b c d  <<< $(echo  "obase=256 ; $1" |bc)
echo ${a#0}.${b#0}.${c#0}.${d#0}

Here they are in action:

google.com has address 216.58.216.142
$ ./ip2d.sh 216.58.216.142
3627735182
$ ./d2ip.sh 3627735182
216.58.216.142

mythtv - Auto-delete duplicate recordings with myth-remove-duplicates.sh

Question:

I have a large archive of recordings on a well-used mythbox and I know I have accumulated a lot of duplicates. I need to free some space, but going through and manually getting rid of the duplicates would be a monumental task.

Is there any mechanism or script available to track down these duplicates, and preferably delete all but the newest recording? I've searched the web and poked at my interface with no luck.

Answer:

You're welcome to use the script I wrote; I've been running it regularly to remove the duplicates.

$ wget http://evuraan.info/evuraan/stuff/myth-remove-duplicates.sh.txt -O myth-remove-duplicates.sh
$ chmod +x myth-remove-duplicates.sh 

Edit to replace RECORDINGDIR, user_name and pass_word with what's applicable to your setup and run myth-remove-duplicates.sh.

Running it :

$ ./myth-remove-duplicates.sh
1091_20131221090100.mpg is duplicate
removed `/var/lib/mythtv/recordings/1091_20131221090100.mpg'
1091_20131221070100.mpg is duplicate
removed `/var/lib/mythtv/recordings/1091_20131221070100.mpg'

Here's the script, myth-remove-duplicates.sh:

#!/bin/bash 
# Authored by Evuraan_AT_gmail_DOT_com
# ABSOLUTELY NO WARRANTY, to the extent permitted by
# applicable law.
# YMMV.
# Use at your own risk.

user_name="mythtv"
pass_word="yourpassword" 
RECORDINGDIR="/var/lib/mythtv/recordings"
list="/tmp/recordings.txt-$RANDOM-$RANDOM"
SQLSCRIPT="/tmp/recordings.sql-txt-$RANDOM-$RANDOM"

gen_lists(){
mysql -u "$user_name" -p"$pass_word" -e "select starttime,basename,title,description from recorded order by starttime" mythconverg | tac > $list
}

verify_duplicate(){
# if verbatim repeats..
sum_a=$(egrep "${a:0:16}" $list | awk -F".mpg\t" 'NR>1 {print $NF}' |md5sum) 
[ ! -z "$sum_a" ] && egrep "${a:0:16}" $list | awk -F".mpg\t" 'END {print $NF}' | md5sum |egrep -q  ${sum_a//-}
}

remove_duplicates(){
[ -s "${RECORDINGDIR}/${a}" ] && ( rm -v "${RECORDINGDIR}/${a}" ; 
	echo "DELETE FROM mythconverg.recorded WHERE basename = '$a';" > $SQLSCRIPT
	)
[ -s $SQLSCRIPT ] && mysql -u "$user_name" -p"$pass_word"   mythconverg < $SQLSCRIPT
}




gen_lists
awk {'print $3'} $list  |grep mpg$ | while read a ; do 
	[[ $(egrep -c ${a:0:16} $list ) -ge 2 ]]  && verify_duplicate && sed -i /"${a:0:16}"/d $list && echo $a is duplicate && remove_duplicates
done

rm $SQLSCRIPT $list 1>/dev/null 2>&1 || : 

How to ensure Set-Top-Box (STB) is Powered On?

.

How to ensure Set-Top-Box (STB) is Powered On?


STB - in most cases, Set Top Boxes are the boxes the Cable/DishTV companies leave at your place with a remote control for you to flip channels. STBs are external, and in case of models like mine, there appeared to be no reliable method to ensure that it is powered on (Like in case of small power disruption. The computer would probably boot back up, but what about the dumb STB?. Read more about the uncertainity here in this thread. Or, here.)

I initially proposed something naive like this (alt link here) and went on to finalize this working script.

Requisites:
You will need ffmpeg and jp2a installed. On my Ubuntu 7.10, they were installed by running:

sudo apt-get install jp2a ffmpeg

check_stb (link)

#! /bin/bash # Authored by Evuraan_AT_gmail_DOT_com # ABSOLUTELY NO WARRANTY, to the extent permitted by # applicable law. # YMMV. # Use at your own risk. pvr="/dev/video0" t="$RANDOM-$RANDOM" out="/tmp/$t.mpg" mark="1727" blank="0" say="/usr/bin/logger -p info -t mythstb-check" POWEROFF="0" POWERON="0" wakeup="/usr/local/bin/irsend SEND_ONCE blaster1 POWER" info="/usr/local/bin/irsend SEND_ONCE blaster1 DISPLAY" send_power_on() { $say "`date` STB seems to be turned off, last count was $blank, attempting to Power it on" & $wakeup 1>/dev/null 2>/dev/null || : sleep 2 } scan_images(){ for i in `echo /tmp/$t*.jpeg` do if [ -s "$i" ] then #blank="`jp2a $i --invert --size=72x24 | sed -e 's/./& \n/g'|grep -c "M"`" blank="`jp2a $i --invert --size=72x24 | sed 's/W/M/g' | sed -e 's/./& \n/g'|grep -c "M"`" rm -f "$i" 1>/dev/null 2>/dev/null || : else : fi [ "$blank" -gt "$mark" ] && export POWERON="100" [ "$blank" -le "$mark" ] && export POWEROFF="100" done if [ "$POWERON" -eq "100" -a "$POWEROFF" -ne "100" ] then # its off send_power_on else # power is on, non blank frames $say "`date` STB seems to be turned on, last count was $blank" : fi } purge_stuff(){ [ -f "$out" ] && rm -f "$out" 1>/dev/null >/dev/null || : } #ivtv-tune -c 3 -d "$pvr" 1>/dev/null 2>/dev/null & cd /tmp $info & dd if="$pvr" of="$out" bs=64K count=2 1>/dev/null 2>/dev/null if [ -s "$out" ] then ffmpeg -i $out -f image2 -vcodec mjpeg /tmp/$t%d.jpeg 1>/dev/null 2>/dev/null [ -s "/tmp/${t}1.jpeg" ] && scan_images else $say "`date` Unable to query $pvr, as it seems to be in use." : fi purge_stuff &

The Idea:
The idea is pretty simple - read from your ivtv device using dd. ffmpeg then converts them into jpeg files (1 per frame). jp2a analyses each of those frames to check whether they're blank or not.


Implementation:
I placed this in my channel change script (example ), so mythtv ensures that the STB is powered on before changing channels. You can also cron it, but it is better to call this in the beginning of your channel change script, as that way, you have a higher chance of unhindered read access to $pvr.

This can be downloaded to your machine as:
wget http://evuraan.info/evuraan/check_stb.txt  -O /tmp/check_stb

To suite specific needs/situations, one may accordingly need to change the parameters pvr, mark and wakeup.

Feedbacks are welcome.