Possible bug in Web Log display

Discussion in 'Tomato Firmware' started by alt12, Jul 13, 2012.

  1. alt12

    alt12 Serious Server Member

    I use Shibby Tomato version on Asus Rt-N12B1.
    I see a problem with the web log display. It's repeatable like this:
    Look at the log for some time period today. Take a screen shot.
    Next day, expand the log to show 100 or 200 entries, and try to review "yesterday's" activities.
    As I compare the screen shot to the log they don't match.
    First time I noticed it was when entire section was missing.
    Today, one or two things are common, then there's a gap again.

    It's important if you need to check exactly where you were at some point in time a day or two before, as I needed to do to debug something.
  2. koitsu

    koitsu Network Guru Member

    Possibly log rotation is what's doing this? I don't know how much data is in your log, but the default log rotational size limit is 50KBytes. The logs aren't rotated on a timestamp (e.g. daily), but instead based on a size limit.

    A much better way to review logs (especially since you're talking about "debugging something") is to use the CLI. Seriously -- it's amazing anyone would use the GUI for debugging purposes when reviewing logs.

    You can find the logs (including the rotated logs) in /var/log. Example:

    root@gw:/tmp/var/log# ls -l
    -rw-r--r--    1 root     root         10254 Jul 12 22:16 messages
    -rw-r--r--    1 root     root         51234 Jul  8 17:06 messages.0
  3. alt12

    alt12 Serious Server Member

    I thought the purpose of Web log is so that I can confirm, in this instance a day later, when a site was visited.
    Can you give me a link to some instructions where it says to use CLI instead of GUI, please. Having never used CLI, I'm not to keen to use it.
  4. tony550

    tony550 Networkin' Nut Member


    I've not yet seen what you are describing, but I do know that
    - the GUI should be fixed to work correctly if it doesn't.
    - the web log is not rotated and is not stored in /var/log .

    Are the entries being sorted correctly?
    Can you correlate the gaps in the log with anything you were doing to the router?
    How many entries did you configure it to remember?
  5. koitsu

    koitsu Network Guru Member

    I took "debugging" in your statement to mean you were actually trying to debug a problem with a daemon/service on TomatoUSB, so that you could fix the underlying code.
  6. alt12

    alt12 Serious Server Member

    I don't know if this is a general Tomato problem or just Shibby's version, but a look-back problem it is.

    After seeing the replies, my log entries from that day were already gone, so I tried replicating.
    Well, I couldn't at first. Comparing few screenshots of a log taken soon after activities to a day later state was all fine...
    Till I switched the number of Weblog entries back to 50 because that was the initial setting when I first saw the missing entries as I expanded to see more of the log because I really needed to prove I was on one website at a certain time, and the log was missing, arrrggghhh.

    Today, I looked at the screenshots taken yesterday. Also yesterday I selected 50 to be displayed. Today, with 50, I could not see the entries, so I changed to 100, still no go. Change to 200 took me to the time slots of interest.

    Bingo - replicated missing entries - Please see three pictures of evidence, with a and c being most severe

    I suspect it has to do with the switching from 50 to a larger number that things break. I'll leave it at 200 in case I need to look back again.
  7. Dark_Shadow

    Dark_Shadow LI Guru Member

    Try downloading the log and compare, I don't have that little bit of traffic on my network to help, setting mine to 200 only shows me a 3 hour window of traffic on my network.
  8. alt12

    alt12 Serious Server Member

    Are you REALLY suggesting that a downloaded log will show different data than side-by-side screenshots? If yes, I'd consider it a major bug, not just an inconvenience for debugging visited websites.

    That said, I did download few logs. It's tough sledding having to convert Linux UTC time stamps. Takes much too much effort. And the few segments I looked at a while back showed no different from what's in GUI.
  9. koitsu

    koitsu Network Guru Member

    Regarding the timezone issue: those logs should be shown in whatever timezone is selected under Basic -> Time. If that isn't the case, I would absolutely classify that as a bug. The entire point to the Time Zone setting is for things to use it. :)

    I do not enable Web Monitoring so I do not have any usage under Web Usage. The logs I was referring to in my previous comment are the ones under Status -> Logs. Sorry about that.
  10. alt12

    alt12 Serious Server Member

    When I download the log, this is how it looks
    So I have to translate the timestamp. Which is why screen shots came handy to see and illustrate the discrepancies.

    Don't apologize. I checked what you might be refering to and came to the conclusion it was about status->logs which show startup and DHCP jobs. So no problem. I'm new to Tomato (used Hyperwrt before) but everything is running well. This log review issue is, for me, the first problem.
  11. koitsu

    koitsu Network Guru Member

    The timestamps shown look like UNIX epoch timestamps, which means they could be based on timezone or based entirely on UTC. UNIX has two functions for this, available on all OSes: gmtime() and localtime(). The former is UTC, the latter is based on the selected timezone. So it all depends on which function is used by both the code that does the actual Web Usage logging itself, as well as the Web Usage log display code. More on that in a moment.

    Here's what I'm talking about:

    $ perl -e 'print scalar gmtime(1342448828), "\n"'
    Mon Jul 16 14:27:08 2012
    $ perl -e 'print scalar localtime(1342448828), "\n"'
    Mon Jul 16 07:27:08 2012
    Both functions are given the same timestamp, but the former shows the timestamp in UTC, while the latter shows it in my local timezone (presently PDT (UTC-0700)).

    If you want an example using strftime() for cleaner formatting I can provide that too.

    IMO, using gmtime() is the better choice because gmtime() isn't susceptible to DST (Daylight Saving Time), but if so this needs to be documented somewhere so that people know when downloading the logs that they'll need to add or subtract their UTC offset (in seconds) from the number.

    Alternately one could use localtime() but a new field (preferably the 2nd field) would need to be added providing what the exact UTC offset is for the local timezone (as this will change in areas with DST). E.g. a log line would need to look like:

    1342448828 -0700 an.avast.com
    Furthermore, if the code does use gmtime() already, then what's shown in the web browser when visiting Status -> Web Usage should be the local timezone (e.g. what's selected in Time Zone), which means the underlying code which shows the timestamp/date should use localtime(), while the code that stores the timestamp itself should use gmtime(). My above perl example is, well, a perfect example of how to do it.
  12. alt12

    alt12 Serious Server Member


    1. I wish that one of the busy Tomato developers (Shibby?, Toastman?) would join in, look at my pictures, and, if changes are indeed needed, rather than it being my error of some sort, also address the UTC time, below.

    2. Using Epoch Converter from the web gave me what I need but it was a major pain. A function you're suggesting would be great - either the version listing local time according to the Time setting on the router, or the version with offset is fine by me.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice