Virgin Media UK cable internet connections are subject to a traffic management policy (commonly referred to as STM) that cuts your bandwidth by 50-75% if you download/upload a certain amount of data between certain hours of the day, as described here. When this happens, your QOS settings become useless and your connection usually slows to a crawl.
Over the last week or so I've been working on a script to monitor bandwidth usage (via rstats) and automatically reconfigure QOS if I go over my STM limit. It seems to work ok, so I thought I'd share it to see if anyone else was interested.
I've set up a small web page here describing it and providing instructions.
The script itself is available here.
You can enable the script on your router by adding the following to your "WAN Up" script and rebooting:
Where [broadband type] = M, M+, L or XL
Here's the first version of the script, just so you can see what's involved:
Even if you're not interested in this particular functionality, there may well be stuff in the script that you find useful for other scripts. I learned quite a lot while writing it.
Any feedback or suggestions most welcome.
Over the last week or so I've been working on a script to monitor bandwidth usage (via rstats) and automatically reconfigure QOS if I go over my STM limit. It seems to work ok, so I thought I'd share it to see if anyone else was interested.
I've set up a small web page here describing it and providing instructions.
The script itself is available here.
You can enable the script on your router by adding the following to your "WAN Up" script and rebooting:
Code:
wget -O /tmp/stm-monitor.sh http://www.tievolu.co.uk/stm/stm-monitor.sh
chmod 755 /tmp/stm-monitor.sh
cru a STM-Monitor "1-59/2 * * * * /tmp/stm-monitor.sh [b][broadband type][/b]"
logger -t STM-Monitor "Downloaded `head /tmp/stm-monitor.sh | grep \"STM Monitor\" | sed 's/# //g'`"Where [broadband type] = M, M+, L or XL
Here's the first version of the script, just so you can see what's involved:
Code:
#!/bin/sh -x
#
# Tomato STM Monitor v1.00
# Written by tievolu
# http://tievolu.googlepages.com/stm-monitor.html
#
# This script is for use on Virgin Media UK cable internet connections,
# which implement Subscriber Traffic Management (STM) i.e. the WAN
# bandwidth is limited for a set time when an predefined amount of data
# is transferred between certain hours of the day.
#
# It allows the WAN bandwidth usage to be automatically monitored and
# suitable QOS settings applied if an STM limit is exceeded. When the
# STM sentence has been served the QOS settings return to normal.
#
# Add the following to your "WAN Up" script to download the script and
# set it to run every two minutes:
#
# wget -O /tmp/stm-monitor.sh http://tievolu.googlepages.com/stm-monitor.sh
# chmod 755 /tmp/stm-monitor.sh
# cru a STM-Monitor "1-59/2 * * * * /tmp/stm-monitor.sh [broadband type]"
# logger -t STM-Monitor "Downloaded `head /tmp/stm-monitor.sh | grep \"STM Monitor\" | sed 's/# //g'`"
#
# Where [broadband type] = "M", "L" or "XL" (no quotes)
#
# Events are recorded in the system log, and details are displayed in a
# small web page located at:
#
# http://[router IP address]/ext/stm-monitor.htm
#
# The script assumes that the time on your router is accurate to within
# a minute or so, so make sure your NTP settings are correct! It also only
# works correcly if your router is left on throughout each STM period,
# because rebooting destroys the Tomato bandwidth stats this script relies on.
# Note that the actions of this script (i.e. setting QOS to deal with an STM
# sentence) use NVRAM variables so they *will* survive a reboot (but not a
# power cycle, because the NVRAM variables are not committed to flash).
# Logging command
LOGGER="logger -t STM-Monitor"
# STM Monitor version
stm_monitor_version=`head /tmp/stm-monitor.sh | grep "STM Monitor" | sed 's/# //g'`
# Check command line argument
if [[ "$#" -ne 1 && "$1" != "M" && "$1" != "L" && "$1" != "XL" ]]
then
$LOGGER "Usage: stm-monitor.sh [M|L|XL]"
exit
fi
# The upload QOS bandwidth values are set at ~91% of the cable modem
# bandwidth limiter (download set at 100%), which works well for my
# connection. Cable connections tend to be pretty stable so they should work
# ok for most users. The STM related variables are based on the information
# published by VM here: http://allyours.virginmedia.com/html/internet/traffic.html
# STM period definitions (time values are hours of the day e.g. 16 = 16:00 = 4pm)
stm_period_name_1="Daytime Downstream"
stm_period_start_1=10
stm_period_end_1=15
stm_period_direction_1="RX"
stm_period_name_2="Evening Downstream"
stm_period_start_2=16
stm_period_end_2=21
stm_period_direction_2="RX"
stm_period_name_3="Evening Upstream"
stm_period_start_3=15
stm_period_end_3=20
stm_period_direction_3="TX"
# STM sentence length (i.e. how long STM lasts for once it's triggered)
stm_sentence_length_hours=5
stm_sentence_length_seconds=`expr $stm_sentence_length_hours \* 3600`
# Settings specific to XL (20Mb down, 768Kb up)
if [[ $1 = "XL" ]]
then
# STM limits (values in MB)
stm_period_bw_limit_mb_1=6000
stm_period_bw_limit_mb_2=3000
stm_period_bw_limit_mb_3=1400
# QOS settings (values in Kbits/s)
inbound_bandwidth_without_stm=20480
inbound_bandwidth_with_stm=5120
outbound_bandwidth_without_stm=700
outbound_bandwidth_with_stm=174
fi
# Settings specific to L (10Mb down, 512Kb up)
if [[ $1 = "L" ]]
then
# STM limits (values in MB)
stm_period_bw_limit_mb_1=2400
stm_period_bw_limit_mb_2=1200
stm_period_bw_limit_mb_3=700
# QOS settings (values in Kbits/s)
inbound_bandwidth_without_stm=10240
inbound_bandwidth_with_stm=2560
outbound_bandwidth_without_stm=465
outbound_bandwidth_with_stm=116
fi
# Settings specific to M (2Mb down, 256Kb up)
if [[ $1 = "M" ]]
then
# STM limits (values in MB)
stm_period_bw_limit_mb_1=1000
stm_period_bw_limit_mb_2=500
stm_period_bw_limit_mb_3=200
# QOS settings (values in Kbits/s)
inbound_bandwidth_without_stm=2048
inbound_bandwidth_with_stm=1024
outbound_bandwidth_without_stm=233
outbound_bandwidth_with_stm=116
fi
# Set up web page. Use a string to store web page content and pipe
# it to /var/wwwext/stm-monitor.htm when the script exits
if [[ ! -e "/var/wwwext" ]]
then
mkdir /var/wwwext
fi
web_page=""
# Grab the current time
current_hours=`date +%H`
current_minutes=`date +%M`
current_unix_time=`date +%s`
# Check whether we are still serving an STM sentence that has completed
if [[ "`nvram get stm_sentence_active`" = 1 && $current_unix_time -gt "`nvram get stm_sentence_end_unix_time`" ]]
then
# Return QOS configuration to non-STM settings
$LOGGER "STM sentence served: applying normal QOS configuration"
nvram set qos_obw=$outbound_bandwidth_without_stm
nvram set qos_ibw=$inbound_bandwidth_without_stm
# Restart QOS
service qos restart
# Unset our NVRAM variables
nvram unset stm_sentence_active
nvram unset stm_sentence_end_time
nvram unset stm_sentence_end_unix_time
fi
# Grab current statistics from rstats and wait for the file to be written
killall -USR1 rstats
sleep 2
stats_file=/var/spool/rstats-speed.js
# Read the stats file and extract the parts we're interested in (vlan1 RX and TX)
read_next_rx_tx=0
while read line
do
case $line in
*vlan1*) read_next_rx_tx=1 ;;
esac
if [[ $read_next_rx_tx = 1 ]]
then
case $line in
*rx:*) rx_data=$line ;;
*tx:*) tx_data=$line ; read_next_rx_tx=0 ;;
esac
fi
done < $stats_file
rm $stats_file
# Check which STM periods are active, check total TX/RX against
# STM limits and note if an STM limit was exceeded
stm_sentence_triggered=0
web_page="${web_page}<table cellpadding=9 cellspacing=3 border=0>"
web_page="${web_page}<tr align=\"center\" bgcolor=\"DDDDFF\"><th>STM Description<th>Start</th><th>End</th><th>Bandwidth Limit</th><th>Bandwidth Used</th><th>Average Rate</th></tr>"
for i in 1 2 3; do
stm_period_name=`eval echo "\\$stm_period_name_$i"`
stm_period_start=`eval echo "\\$stm_period_start_$i"`
stm_period_end=`eval echo "\\$stm_period_end_$i"`
stm_period_direction=`eval echo "\\$stm_period_direction_$i"`
stm_period_bw_limit_mb=`eval echo "\\$stm_period_bw_limit_mb_$i"`
if [[ $current_hours -ge $stm_period_start && $current_hours -lt $stm_period_end ]]
then
web_page="${web_page}<tr bgcolor=\"EEEEEE\" align=\"center\">"
web_page="${web_page}<td align=\"left\">${stm_period_name} (${1})</td><td>${stm_period_start}:00</td><td>${stm_period_end}:00</td><td>${stm_period_bw_limit_mb} MB</td>"
# Calculate the number of values to take from the rstats data (each value covers two minutes)
minutes_since_stm_period_started=`echo "" | awk '{ print((((hours - stm_start) * 60) + minutes)); }' hours=$current_hours minutes=$current_minutes stm_start=${stm_period_start}`
number_of_values=`expr $minutes_since_stm_period_started / 2`
if [[ $stm_period_direction = "RX" ]]
then
data=$rx_data
else
data=$tx_data
fi
if [[ $number_of_values != 0 ]]
then
# Sum the last XXX values of the 720 in the rstats data to get the
# total bytes transferred since the STM period started
total_transferred_bytes=`echo $data | awk '
{
values_string = substr($0, index($0, "[") + 1, index($0, "]") - 6);
split(values_string, values_array, ",");
for (i=(721-number_of_values); i<=720; i++) {
total += values_array[i];
} printf("%0.f" total);
}' number_of_values=$number_of_values total=0`
total_transferred_mb=`echo "" | awk '{ printf("%.2f", bytes / 1048576) }' bytes=$total_transferred_bytes`
average_rate=`echo "" | awk '{ printf("%.2f", bytes / (number_of_values * 2 * 60 * 1024)) }' number_of_values=$number_of_values bytes=$total_transferred_bytes`
else
total_transferred_bytes="0"
total_transferred_mb="0.00"
everage_rate="0.00"
fi
# Calculate maximum average rate (i.e. rate required to trigger STM),
# then compare this to our actual rate. If the actual rate is equal or
# higher, we paint the cell red, if it's greater that 75% of the max
# we paint it amber, and if it's anything else we paint it green.
stm_period_max_ave_rate=`echo "" | awk '{printf((limit * 1024) / ((end - start)*3600))}' limit=$stm_period_bw_limit_mb start=$stm_period_start end=$stm_period_end`
cell_color=`echo | awk '{ if (rate >= max) {print("FF6633")} else if (rate > (0.75 * max)) {print("FFCC33")} else {print("66FF33")}}' rate=$average_rate max=$stm_period_max_ave_rate`
# Add data to web page
web_page="${web_page}<td>$total_transferred_mb MB</td><td bgcolor=\"${cell_color}\">$average_rate KB/s</td>"
# Check whether STM limit has been exceeded
stm_period_bw_limit_bytes=`expr $stm_period_bw_limit_mb \* 1048576`
result=`echo "" | awk '{ if (total > limit) {print(1)} else {print(0)}}' total=$total_transferred_bytes limit=$stm_period_bw_limit_bytes`
if [[ $result = 1 && "`nvram get stm_sentence_active`" != 1 ]]
then
stm_sentence_triggered=1
nvram set stm_sentence_active=1
$LOGGER "STM Triggered: ${stm_period_name} (${stm_period_start}:00 -> ${stm_period_end}:00, ${stm_period_direction}, ${stm_period_bw_limit_mb} MB)"
fi
else
web_page="${web_page}<tr bgcolor=\"EEEEEE\" style=\"color:AAAAAA\" align=\"center\">"
web_page="${web_page}<td align=\"left\">${stm_period_name} (${1})</td><td>${stm_period_start}:00</td><td>${stm_period_end}:00</td><td>${stm_period_bw_limit_mb} MB</td><td>N/A</td><td>N/A</td>"
fi
web_page="${web_page}</tr>"
done
web_page="${web_page}</table>"
# If STM is active, check whether QOS is set accordingly
if [[ "`nvram get stm_sentence_active`" = 1 && "`nvram get qos_obw`" != $outbound_bandwidth_with_stm ]]
then
if [[ $stm_sentence_triggered = 1 ]]
then
# We must have triggered an STM sentence on this run.
# Work out the hour component of the time the STM sentence will end
# (for logging purposes only - we will use unix time to track the
# STM sentence accurately)
stm_end_hour=`expr $current_hours + $stm_sentence_length_hours`
if [[ $stm_end_hour -gt 23 ]]
then
stm_end_hour="0`expr $stm_end_hour - 24`"
fi
# Set NVRAM variables to describe the STM sentence being served
nvram set stm_sentence_end_time="$stm_end_hour:$current_minutes"
nvram set stm_sentence_end_unix_time=`expr $current_unix_time + $stm_sentence_length_seconds`
# Send information to log
$LOGGER "STM sentence started: applying STM QOS configuration (until $stm_end_hour:$current_minutes)"
else
# An STM sentence is active and we didn't trigger it on this run,
# but the QOS settings are wrong. We must have rebooted, or someone
# messed with the QOS settings. So we want to set the QOS settings
# to deal with the STM, but we don't want to overwrite our NVRAM
# variables.
# Send information to log
$LOGGER "STM sentence resumed: applying STM QOS configuration (until `nvram get stm_sentence_end_time`)"
fi
# Set QOS NVRAM variables
nvram set qos_obw=$outbound_bandwidth_with_stm
nvram set qos_ibw=$inbound_bandwidth_with_stm
# Restart QOS
service qos restart
fi
# Add QOS settings to web page
web_page="${web_page}<p>Current QOS settings - <b>`nvram get qos_ibw`</b> kbits/s down, <b>`nvram get qos_obw`</b> kbits/s up</p>"
# Add latest log entries to web page
web_page="${web_page}<h3>Latest Log Entries</h3>"
log=""
while read line
do
case $line in
*STM-Monitor*) log="${log}<nobr><tt>${line}</tt></nobr><br>" ;;
esac
done < /tmp/var/log/messages
web_page="${web_page}$log"
# Publish web page
# Write the title and status now, so that they accurately relfect the data in the table
web_page_header="<html><head><title>${stm_monitor_version}</title><meta http-equiv=\"refresh\" content=\"120\"</head>"
web_page_header="${web_page_header}<body style=\"font-family:verdana\"><h2>${stm_monitor_version}</h2>"
web_page_header="${web_page_header}<h3>Status at ${current_hours}:${current_minutes} - "
if [[ "`nvram get stm_sentence_active`" = 1 ]]
then
web_page_header="${web_page_header}<b style=\"color:red\">STM has been triggered - the sentence will end at `nvram get stm_sentence_end_time`</b></h3>"
else
web_page_header="${web_page_header}<b style=\"color:green\">STM has not been triggered</b></h3>"
fi
web_page="${web_page_header}${web_page}</body></html>"
echo $web_page > /var/wwwext/stm-monitor.htmEven if you're not interested in this particular functionality, there may well be stuff in the script that you find useful for other scripts. I learned quite a lot while writing it.
Any feedback or suggestions most welcome.
