1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Memory Leak

Discussion in 'Tomato Firmware' started by Riddlah, Aug 14, 2012.

  1. jyavenard

    jyavenard Network Guru Member

    so what was it ?
     
  2. GhaladReam

    GhaladReam Network Guru Member

  3. shibby20

    shibby20 Network Guru Member

    is that problem exist also in tomato-RT? Or maybe only on RT-N?
     
  4. jyavenard

    jyavenard Network Guru Member

    Well, unless you tell us what the problem is/was. How could you know for sure?

    No sure what the secrecy over this is for...
     
    koitsu likes this.
  5. Cyberian75

    Cyberian75 Network Guru Member

    Isn't that too much? Doesn't the router need more RAM for its operation?
     
  6. shibby20

    shibby20 Network Guru Member

    @jya its not a secret and i want to push changes to git but i`m not sure... should i make changes only in tomato-RT-N or mayge in tomato-RT also.

    i made update`s from asus sources for files:
    -driver/net/pppol2tp.c
    -driver/net/pptp.c
    -net/ipv6/sit.c

    i update also radvd +libdaemon to latest version
    i update also a code od support 64K nvram for RT-N66U.

    new builds:
    RT-N16 and others K26USB MIPSR2 routers:
    http://update.groov.pl/ipv6/tomato-K26USB-1.28.RT-MIPSR2-100.6-BT-VPN.trx

    RT-N66U with new 64K code:
    http://update.groov.pl/ipv6/tomato-K26USB-1.28.RT-N5x-MIPSR2-100.7-Mega-VPN-64K.trx
    AIO will be for a few minutes.

    No new to clean nvram i you have already 64k mod.

    Best Regards.
     
    Cyberian75 likes this.
  7. leandroong

    leandroong Addicted to LI Member

    Suggestion, don't rush, concentrate on RT-N and wait feedback.
     
  8. shibby20

    shibby20 Network Guru Member

  9. Cyberian75

    Cyberian75 Network Guru Member

    I'm glad to report that leaks have been sealed. Good job, Shibby! :)
     
  10. GhaladReam

    GhaladReam Network Guru Member

    100.7 AIO 64k working here as well :)
     
  11. koitsu

    koitsu Network Guru Member

    Can someone provide/upload actual diffs for these (compared against the version/build which was leaking)? I'd like to see exactly what got changed in this underlying kernel driver code.

    Furthermore, I imagine Toastman is going to need the exact same changes/diffs.
     
  12. shibby20

    shibby20 Network Guru Member

    I will push changes to git tomorrow
     
    pharma likes this.
  13. jyavenard

    jyavenard Network Guru Member

    All of those are in the RT branch too. So any problems in the RT-N branch is also in the RT one.

    If you have done what I think you have done (fix that is also in the 0501.4 build I posted above) then it's a very poor solution and will break IPSec support that you recently added.

    At the end of the day, it's always a worry when you have a non coder starting to patch the Linux kernel... Do you even know what you are doing?
     
  14. koitsu

    koitsu Network Guru Member

    Without turning this thread into a battle or flame war, as someone who has worked in open-source communities since the late 90s, I can understand where jyavenard is coming from here. And I say that with full/total respect to Shibby and his efforts to get a fix in place for the problem (honestly! It's good to have firmware maintainers that are attentive!). I have no beef with anyone on here.

    But I believe jyavenard's point is that he's concerned that the fix in question may have actually induced more problems, and now we have 1) people using firmwares with this fix (and it's good that it works for them) but without a full/good amount of testing to see if it breaks other things, and 2) a commit that's coming into the tree that could cause problems if (a) is not addressed (i.e. confirmed or denied) first.

    For example, I would probably be a bad candidate for tinkering about in the Linux kernel since my Linux experience is limited, but I have spent time in the FreeBSD kernel and *way* too much time in userland (both as a user and as a programmer/author of software), which is how my earlier "debugging" methods were of use. This is one of the reasons I wanted to see diffs BEFORE the commit (vs. just commit and then use whatever web interface to git to see the diffs). There may be something in the diff that stands out ("ah, they're missing a couple *free() statements in this one block of code, that should fix it. We don't need the rest of the diff") and thus should be the only thing backported/merged vs. an entire .c file being replaced.

    Testing/QAing is a time-consuming thing, and I say that with full acceptance that the memory leak issue is serious. I think everyone is on the same page WRT the leak being a problem, but making sure that whatever fixes/etc. get imported properly (to ensure nothing else breaks as a result) is equally as important. See leandroong's comment earlier in the thread: re: take your time, don't rush it.

    So I don't read jvayenard's comment to be some sort of totalitarian statement, I read it as "uh, are you sure you're doing the right thing here? Because otherwise I'm worried this is going to cause breakage elsewhere. Let's work together please". That's also indicated from his writing tone in this post.

    But then again maybe it's just because I'm used to generally "prickly" (I didn't say prick, I said prickly) personalities in the open-source world -- because I have that exact same personality. There's a lot more of that kind of thing in the BSD world than there is in the Linux world, let me tell you. :)
     
    pmd and jyavenard like this.
  15. Dark_Shadow

    Dark_Shadow Addicted to LI Member

    100.7 AIO 64k working here as well, Although I don't use ipspec or ipv6 yet everything else seems to be working fine.
     
  16. Cyberian75

    Cyberian75 Network Guru Member

    Well, he didn't really "release" it yet. We're his testers, and that's what we're doing; after all, he stated "feedback welcome" which is equivalent to "test and let me know."
     
  17. Malitiacurt

    Malitiacurt Networkin' Nut Member

    You guys are putting Shibby on a high pedestal cause he creates the releases, but the behind-the-scenes people do the thankless work.

    For example, do you know who made a bunch of the Multi-SSID fixes to Tomato recently, or ported the 64k nvram code to the RT-N66U (which DD-WRT still has not fixed), or many of the driver/kernel level fixes? Jyavenard did. You can follow a bunch of his work at TomatoUSB and has been developing features and fixes for both Shibby and Toastman.

    Leave the guy alone, he has a valid point since he's been the one doing the core level fixes, shibby normally just integrates different code sources, adds utility features, and makes the releases.
     
    koitsu likes this.
  18. pharma

    pharma Network Guru Member

    Maltiacurt,

    Get the facts straight. While Jya is one of the developers he is not the only one doing core level fixes. and the others seem to have less of an ego. It's one of the reasons many are hoping Jon or Teddy Bear will make an appearance, though unlikely. A list of features incorporated into Toastman, excluding many enhancements made to other individual mods by their respective developer's, and not including other people who have contributed "passively" to enhancing Tomatos capabilities:

     
    koitsu likes this.
  19. leandroong

    leandroong Addicted to LI Member

    We should be thankful for both:
    1. Shibby, for fixing this memory leak issue. I doubt it to be fixed soonest if not for him
    2. jyavenard, for fixing the "ugh..." thing issue that consider to be major issue also
    I'm happy seeing them combining their skills. It speedup tomato development. Now, I wonder, who will help us get dual wan working? Maybe, merlin can join in as well?
     
  20. jyavenard

    jyavenard Network Guru Member

    I'm sorry it came across that way... Shibby is the first one to state that he's not a programmer.

    Anyhow, from what Shibby has hinted, I see what he's done. He has introduced changes made by Asus to their latest kernel. Those changes are on
    -driver/net/pppol2tp.c
    -driver/net/pptp.c
    -net/ipv6/sit.c

    And they have modified the way the RT-N66 64K behaves when upgrading a RT-N66 with a 32kB nvram.

    If you look at Shibby's earlier post, you see that that's exactly what has been modified.

    I know for a fact that if you made the changes Asus made to net/ipv6/sit.c to fix the leak (which is a change I added earlier http://linksysinfo.org/index.php?threads/memory-leak.38924/page-4#post-187798)
    You will break IPSec support that Shibby recently committed. Asus doesn't have IPSec support in their kernel so they don't care about it.

    At the end of the day, all I wanted was to Shibby to share his findings, the best would be for Shibby to create a branch on git with those changes, so they can be properly reviewed before being integrated in his main build.
    This is an open source project, let's work on it like we should: be open

    I've ordered a RT-N66 (I'm away from how for two months, so couldn't look/test before) will be able to work on it properly then.
     
  21. jyavenard

    jyavenard Network Guru Member

    You aren't using IPv6 and you were seeing the leak ? now that's something different
     
  22. jyavenard

    jyavenard Network Guru Member

    Not wanting to get into a mine is bigger contest. But that list is not very relevant at listing who's done what. It only list the original web interface features and none of the fixes that were done over time, and (more importantly) none of what is going under the hood.

    There has been plenty of people providing fixes on all those features, yet you do not see their names there.
     
    koitsu likes this.
  23. Dark_Shadow

    Dark_Shadow Addicted to LI Member

    No, No leak here at all, I was just reporting about not seeing anything else broken by the fixes applied.
     
  24. jyavenard

    jyavenard Network Guru Member

    Ah ok...
    if you aren't using IPv6 or IPSec, then yes... you won't see any problems.
    I haven't looked into details on what PPTP fixes they've added in the Asus kernel .Tomato kernel is more recent and most of their changes are irrelevant to tomato's one.
    I wouldn't have included those changes in tomato's one.
     
  25. pharma

    pharma Network Guru Member

    Shibby,

    Looking forward to running your firmware on the Toastman version. If you want to test your fixes on different routers create a few firmware versions that can be used on a E4200 v1. :)
    Specifically, I'd be interested in trying this on your TOR version.

    Thanks,
    Pharma
     
  26. Elfew

    Elfew Addicted to LI Member

    Excellent! Thank you...
     
  27. Mirko Baila

    Mirko Baila Networkin' Nut Member

    Hi,the firmware 100.7 is stable ,or is it better to wait for a new version?
    Thank's ;)
     
  28. Cyberian75

    Cyberian75 Network Guru Member

    If you don't wanna test it, I'd wait for an official release.
     
  29. Mirko Baila

    Mirko Baila Networkin' Nut Member

    Ok thank's ;)
     
  30. shibby20

    shibby20 Network Guru Member

    @jya your post was very rude. Are you angry on me or maybe just jealous that i tracked the problem and found solution first. Dont get it. Maybe i`m not a programer but i am administrator of linux systems and i have a knowledge about modify and patching kernel sources. Maybe you don`t remember but few months ago i was added support of NAND and WNR3500Lv2. Even ddwrt doesn`t support this router yet. If i really didn`t know what i`m doing well my tomato would be a rubbish and no one will use it.

    FYI
    wrong. You tell us about patch
    https://github.com/RMerl/asuswrt-merlin/commit/56c07eb44d11f40fa9a691a4d4426ce53acd8004
    and i wrote you that tomato already has this patch included
    http://repo.or.cz/w/tomato.git/commit/f4e040a9a69fd07c2b9d4f2e8b0382fc7ac25efa

    in the latest asus sources is different!
    Code:
    --- a/sit.c<--->2011-09-26 09:16:47.000000000 +0200
    +++ b/sit.c<--->2012-08-24 12:40:54.000000000 +0200
    @@ -419,10 +419,10 @@
    <----->}
    .
    <----->/* no tunnel matched,  let upstream know, ipsec may handle it */
    +<----->kfree_skb(skb);
    <----->read_unlock(&ipip6_lock);
    <----->return 1;
    out:
    -<----->kfree_skb(skb);
    <----->return 0;
    }
    
    well you was wrong. Do you see a different?
    BUT FYI no need to make this change, no need to change sit.c I revert this file from git and memory leak is still fixed.

    Patch is big because i update radvd to the latest version and add libdaemon. The patch only with changes in kernel is available here (on RT-N branch only):
    http://tomato.groov.pl/download/K26RT-N/testing/ipv6

    radvd +libdaemon i will push to git later.

    Best Regards!
     
    FlyingHorse and kthaddock like this.
  31. leandroong

    leandroong Addicted to LI Member

    Welcome back Shibby !!! Everybody missing you. Jyavenard, just curious how you did it. He learned it and was scolded by many of your followers. Forgive and forget. Both of you guys are assets.
     
  32. jyavenard

    jyavenard Network Guru Member

    Damn, my long reply got lost, stupid web site.

    Again, I'm sorry it came across that way...

    As far as jealousy goes.. I really couldn't care less...
    As I mentioned earlier, I'm traveling overseas for two months, I have no access to a router I could try anything on.

    It seems to me that you're the only one thinking it's a race in finding the problem, why else wouldn't you share any of the changes, so it can be reviewed by others?

    Should I remind you that you're the one applying changes made by me or others and putting yourself as author. And not just once. Now that would be sufficient to make anyone angry.
    I accepted your explanation that you didn't know how to use git.

    Well, I hope you aren't applying that change to the code, because now that is *very* wrong. You are freeing code even though you let IPSec continue handle the packet later on. return 1 should be removed. Or the handling in IPSec must be changed

    I don't want to start a pissing context, but all those changes were in my earlier post.
    If you check:
    http://www.avenard.org/wrt54-tomato/tomatoUSB/

    You'll see:
    http://www.avenard.org/wrt54-tomato...0501.3-MIPSR2-jya-RT-N-VLAN-VPN-NOCAT-64K.trx
    http://www.avenard.org/wrt54-tomato...0501.4-MIPSR2-jya-RT-N-VLAN-VPN-NOCAT-64K.trx
    http://www.avenard.org/wrt54-tomato...0501.5-MIPSR2-jya-RT-N-VLAN-VPN-NOCAT-64K.trx

    .3: is with radvd + libdaemon: tested by some and found not to fix the leak
    .4: is without the multiSSID fix: tested by yourself and you said it didn't fix the leak. You had mentioned earlier that these commits introduced the IPv6 leak... So what is it ?
    .5: is with the IPv6 sit.c fix. unfortunately, that change hasn't seem to be tested by anyone unfortunately. It would be great if it was tested as it's a one line change in the IPv6 kernel code.

    All of those changes were posted before you wrote that you found the fix.

    And that's where I think your approach is wrong. You go with a chainsaw, applying all the Asus changes made on their last build when we don't know why those changes were needed.
    You need a gentle scalpel approach.
    There's no way you can tell which changes actually fixed the problem when so many things have been touched.

    Applying all the Asus changes, and reverting them one by one until it works isn't the way to fix it either. Because you still don't know the why. It only gives you an idea on where to start looking.

    That you revert the sit.c change to realise the leak is still fixed proves to me that you're not 100% sure which changes fixed it
    Also, no one is able to review your changes.

    http://tomato.groov.pl/download/K26RT-N/testing/ipv6
    includes changes to l2tp and pptp. I see nothing in there that change the way memory is allocated or freed that would explain why a leak is occurring when using IPv6 . Also, how many people are using pptp or l2tp to connect to their ISP and reporting the leak? very few I would assume. Those connection methods are rarely used these days. It's mostly PPPoE or direct IP (with DHCP allocation)

    The problem again, is that you don't share your findings, only few binary images for people to test. This is an open source project. Or so I thought.
     
  33. pharma

    pharma Network Guru Member

    @jya,

    Keep your ego in place! If you have nothing to contribute please shut-up.

    I see you doing the same regarding Cisco fixes, and perhaps fixes from other "passive" developers. Tomato will always be "broken" with regard to specific issues until a solution is found and applied, so no one expects it to be 100% error free. Like all past firmware versions they have all included fixes for issues that have been around for months or years, and not knowingly may have broken other components. It seem this fix has corrected an issue that many here experienced but according to you may break some other component that few people are using.

    I suggest you test the "fixed" firmware against the component you think it will break and show your results here or offline with the pertinent developers. However, what is happening here is no different than what has happened in the past with break fixes --- once applied and is incorporated into the Tomato "beta" firmware is used by the general community until something else "breaks" and is fixed.

    Pharma
     
    FlyingHorse likes this.
  34. shibby20

    shibby20 Network Guru Member

    hmm. Take your RT-N branch. Add radvd+libdaemon(this may be no needed to do but upgrade package is always a good way :) ) and apply my "ipv6" patch and leak will disappear. Maybe not all changes are needed but it`s works. I tested also pptp server after patch and works great. Agree ... maybe it`s not the best solution but IMO (administrator of linux systems) if all works no need to rest more time. I have another problems/things to makes. Programmer can makes code better :)

    disagree

    you wrote:
    well 501.4 has sit.ko change and memory leaks - that i was confirm

    and then you wrote:
    another 501.4?? i think should be 501.5.
    well
    501.4 with sit.ko change (tested by me) - memory leaks
    501.5 without multissid fix - IMO memory leak fixed but i didnt test it.

    it`s not a race between us. I do this for users. My two last builds has memory leaks. I also had to disable ipv6 support. Well i want to find and fix problem as quickly as possible.

    May i have a request for you? Please look into ipv6 patch and tell me/us, is there something wrong. Is this patch broke something? If not i will be happy to add this patch to git and make new v101 for all users. There are a lot of users how want use ipv6 and now they can`t. We can later search a better solution for this.

    I didn`t publish changes before because:
    - i want test it first myself (test build i release only for hardcode users :) )
    - on saturnday i was in work and i haven`t time.
    - afternoon of saturnday i was in my family and i haven`t access to PC. I read forum on my android phone.

    well i have time today make and publish a patch.

    Best Regards
     
    kthaddock likes this.
  35. jyavenard

    jyavenard Network Guru Member

    I'm pretty sure that my contribution to tomato so far has greatly surpassed yours so far...

    what are you on about?
    What Cisco fixes are you talking about ?
     
  36. myersw

    myersw Network Guru Member

    Folks can we stop the mine is bigger then yours??? We the end users of this great firmware know that this is a group effort and are getting tired of the you said but I did stuff. Lets agree that jya and shibby both have given us a great Tomato and lets not forget if it were not for Jon and TeddyBear we would not have this at all.
    --bill
     
  37. jyavenard

    jyavenard Network Guru Member

    the pptp server won't be using that part of the code...

    Anyhow, let's restart from the beginning:

    What exactly were you doing to experience the leak?
    How was your WAN connected? DHCP, PPPoE, L2TP, PPTP ?

    Then what type of IPv6 connection were you using?

    Thanks
    JY
     
  38. shibby20

    shibby20 Network Guru Member

    as i said i`m watching youtube. Then memory leaks really fast

    DHCP and i have got static public ip

    6in4 static tunnel to HE.net
     
  39. pharma

    pharma Network Guru Member

    Myersw, I could not agree more but if seems a "programmer" is over-inflating his importance/contribution to be comparable to Jon and TeddyBear. I think the main issue is someone found a solution (perhaps improvised) that is well beyond the capability of the other, hence all the how-to questions.

    With your history and background with Tomato, I'm glad you piped in ... :)
     
  40. myersw

    myersw Network Guru Member

    pharma, thanks. I have used Tomato since day one I guess. I provided Jon a WRT54G v1.1 that I was not using when he was having issues with that model and did not have one.
    The how to questions are fine with me, its the you don't know anything and I do attitude that I object to.
    --bill
     
  41. RMerlin

    RMerlin Network Guru Member

    Asuswrt-Merlin's DualWAN is actually 100% Asus's code - it's just not enabled by default in their official releases because (for one thing) it's still very much a work-in-progress.

    That code is spread all over the place. Porting it to Tomato would probably not make it much easier than actually developing it from scratch if someone wanted to add it to Tomato. While the two firmwares still share a lot of common design, they have diverted too far apart over the years to make it a simple port from Asuswrt to Tomato.
     
  42. Cyberian75

    Cyberian75 Network Guru Member

    Exactly. And one has to be willing to learn even from an infant.
     
  43. jyavenard

    jyavenard Network Guru Member

    I'm not sure if you're referring to me here nor if I should bother arguing with you on that point..
    All I'm trying to do here is narrow the scope of where to look in all the things shibby has modified.

    That way we can have the smallest change possible that will fix this leak...
     
  44. jyavenard

    jyavenard Network Guru Member

    Ok thanks...

    One more thing... If you disable the 6in4 static tunnel, do you still have the leak?

    The ppol2tp and pptp.c code in Toastman is very different to what's in the shibby branch. Toastman reverted commit http://repo.or.cz/w/tomato.git/commit/cbe90da626146b19786c7a9816d7c0fdaaa4fe44

    The only thing that would explain fixing a leak would be this part

    If that's the case, Toastman branch shouldn't leak.. but it does...

    very weird...

    If Toastman could chip-in and explain why he reverted the l2tp commit that would help
     
  45. Cyberian75

    Cyberian75 Network Guru Member

    Leaks stopped when the tunnel was disabled.
     
  46. jyavenard

    jyavenard Network Guru Member

    Guys... can we get passed all this and concentrate on the matter at hand?

    Again, I apologise if my earlier post was inappropriate...

    In hindsight, I certainly regret it came that way and shouldn't have posted it.

    While it doesn't justify why I posted what I posted, let me give some background: the reason I'm overseas is: my dad passed away last week. It was very late, I had just got back from the funerals, I was tired, depressed, a tad drunk and I just vented off.

    I guess I should have avoided posting anything over the past week....
     
  47. Cyberian75

    Cyberian75 Network Guru Member

    Sorry about your father...
     
  48. myersw

    myersw Network Guru Member

    jya, thanks for giving us some insite as to what was going on with you. Sorry for your lose!
    Know you and the other folks here will figure out the memory leak issue and what needs to be done to correct. Have done some coding and debugging and know it is not an easy task taking someone else's code and figuring out what is wrong. Especially when the simple looking device has some very complicated functions, and different folks make use of different combinations of those functions make lots of fun for the coders to cover all the combinations and not break something.
    --bill
     
  49. guardian

    guardian Serious Server Member

    Just want to mention that I use l2tp to connect to my ISP (with Toastman/JYAvenard brachnes). And I use 6in4 tunnel (he.net) to get IPv6 connectivity. And I have a leak in both branches: latest Toastman's build and build 100 by Shibby, and jyavendard july builds. Turning tunnel off - leak disappears. Just to mention, L2TP in shibby's build is broken for use with my ISP because of a compatibility issue. That brokenness was fixed in Tostman's branch some time ago.
     
  50. Toastman

    Toastman Super Moderator Staff Member Member

    Hmm. That was some time ago .... as I recall, there were issues some time back with the l2tp which were raised by E.European (Russian?) users. I believe this commit broke l2tp for Russian users, so I reverted it to fix the issue. There were, I believe, also some very small changes to the l2tp code in addition which were suggested by some guys in that country. I recall the matter was mostly discussed on tomatousb.org. It seemed to fix the problem for them, and I have never had any reports about it breaking compatibility in other countries (if there are any?)
     
  51. jyavenard

    jyavenard Network Guru Member

    Guardian, what router do you have?
    I'll make a build with some of Shibby's change.
     
  52. kthaddock

    kthaddock Network Guru Member

    jya
    If you want that change, russian I can post when I come home.
     
  53. guardian

    guardian Serious Server Member

    Yes, the problem with l2tp was with russian ISP called "Beeline". L2TP with Beeline is ok with latest Toastman build. Please, please - don't break it again :)
     
  54. guardian

    guardian Serious Server Member

    JYAvenard, I have a testing lab with both Asus RT-N16 and Asus RT-N66U :)
     
  55. jyavenard

    jyavenard Network Guru Member

    Excellent thanks.
    I'm uploading http://www.avenard.org/wrt54-tomato....28.0501.6-MIPSR2-jya-RT-N-VLAN-VPN-NOCAT.trx

    For your RT-N16.

    What's amazing is that this bug only shows up now.. there has been no changes to this part of the kernel in months (> 1 year for shibby branch)

    guardian, you may want to try the latest shibby's build with yet again new changes to the l2tp code, it may very well fix your issue with your ISP. Would be good to know as it would allow that have shibby and Toastman in sync there
     
  56. kthaddock

    kthaddock Network Guru Member

    As I promised:
    Code:
    #!/bin/sh
     
    # getting L2TP WAN ipv4
    IPV4ADDR=`ifconfig ppp0 | grep inet | grep -v inet6 | cut -d ":" -f 2 | cut -d " " -f 1`
    # making ipv6 from ipv4
    IP6ADDR=`echo $IPV4ADDR | awk 'BEGIN{FS="\."}{printf"2002:%02x%02x:%02x%02x:1:0:0:0:1",$1,$2,$3,$4}'`
    # deleting existing tunnel made by TomatoUSB front-end
    ip t d v6to4
    # adding new tunnel
    ip t a v6to4 mode sit remote any local $IPV4ADDR
    ip l s dev v6to4 up
    # routing 6to4
    ip -6 a a $IP6ADDR/16 dev v6to4
    ip -6 route add 2000::/3 via ::192.88.99.1
     
  57. jyavenard

    jyavenard Network Guru Member

    What change would that be? What is it for? Is this for Russian users only?
     
  58. kthaddock

    kthaddock Network Guru Member

  59. koitsu

    koitsu Network Guru Member

    If this is in reference to the memory leak: I'm not too surprised by this. IPv6 adoption has increased in the past 3 months, more and more end-users are starting to care about it. The leak has probably always been there, but just now people are finally reporting the problem. Generic end-users often do not have the technical skill set to determine a memory leak; it took a user stating "I keep noticing my free memory gets lower and lower until my router reboots", meaning a user who was keeping a close eye on things, before it came to light. To me that's not very surprising. I would have noticed it myself except I don't use tunnel services, I use native IPv6.
     
  60. pharma

    pharma Network Guru Member

    I also think many here like myself currently have IPv6 disabled, so all symptoms related to free memory getting lower and the router rebooting never appeared.

    @kthaddock,
    Thanks for the link. I hope rz2k starts posting in this forum because it's unfortunate his post did not receive the proper feedback.

    Pharma
     
    kthaddock likes this.
  61. GhaladReam

    GhaladReam Network Guru Member

    This is the case for me as well. My connection comes in as native IPv4 DHCP from my ISP.
     
  62. USNetboy

    USNetboy Networkin' Nut Member

    I'm doubtful if this is the case. The bug has something to do with the introduction of the new .20 Broadcom driver as @jya indicated a few pages back in post 52 and from my own personal experience. I have been running Toastman-RT I built from latest git pull on 7/14 on my RT-N16. I've had an IPv6 tunnel with HE for more than a year now and the setup was rock solid. I've decided to upgrade to RT-N66U and built Toastman-RTN from latest pull on 8/24 and had the leak show up in full force. This bug is very hard to ignore if you are a 6in4 user. I'm using the "whitelisted" Google DNS forwarding so all my YouTube traffic is running over the tunnel and boy let me tell ya, it leaks big time. During heavy YouTube usage it would crash and reboot every 5-10 minutes will users loosing wireless - very hard to miss.
     
  63. Cyberian75

    Cyberian75 Network Guru Member

    For me, it took less than 5 minutes to reboot.
     
  64. koitsu

    koitsu Network Guru Member

    It's hard to miss if you don't use wireless -- like me. It's also hard to miss when the entire thread -- pages worth -- is filled with a user stating that the problem goes away when IPv6 support (specifically use of IPv6 tunnel) is disabled.
     
  65. guardian

    guardian Serious Server Member

    JYAvenard: installed tomato-K26USB-1.28.0501.6-MIPSR2-jya-RT-N-VLAN-VPN-NOCAT.trx on Asus RT-N16, as you asked. L2TP works. Memory leaks. To observe a fast leak it is enough to start watching a YouTube clip over IPv6 tunnel.
     
  66. jyavenard

    jyavenard Network Guru Member

    Thank you for this report..

    Can you check:
    First:
    http://www.avenard.org/wrt54-tomato....28.0501.7-MIPSR2-jya-RT-N-VLAN-VPN-NOCAT.trx
    And after:
    http://www.avenard.org/wrt54-tomato....28.0501.8-MIPSR2-jya-RT-N-VLAN-VPN-NOCAT.trx

    Also, if you could test Shibby latest build with his IPv6 fix, mainly to test if the newer L2TP now works for you...
    If he hasn't published it, I can build it for you
    JY
     
  67. guardian

    guardian Serious Server Member

    JYavenard: could you please build a old-good-non-64Kb-nvram AIO build? It would be easier for me to test because I have config backups ready for shibby non-64kb branch.

    Will try .7 and .8 and report.
     
  68. jyavenard

    jyavenard Network Guru Member

    Will do... with the IPv6 patch...
     
  69. guardian

    guardian Serious Server Member

    JYAvenard:

    on Asus RT-N16:
    .7: leaks
    .8 do not leak
    waiting for Shibby branch build for test.

    Do you want me to test .8 on RT-N66?
     
  70. jyavenard

    jyavenard Network Guru Member

    .7 is without multiSSID fixes I made last month that Shibby said caused the leak
    .8 is with the older driver

    This confirmed what I thought all along and described right at the beginning: that it's a regression introduced in the new drivers... Unfortunately, we do not have the source code of those...

    The more it goes, the more I doubt Shibby's IPv6 patch will fix anything, provided it doesn't touch IPv6 kernel code at all...
    But will see.

    it's building now.... I don't have a very fast machine... it takes a while

    No need... 64K or not, the problem would be the same.
     
  71. jyavenard

    jyavenard Network Guru Member

  72. guardian

    guardian Serious Server Member

    JYAvenard:

    tomato-K26USB-1.28.0501.9-MIPSR2-jya-shibby-RT-N-AIO.trx: L2TP _do_not_ work, will try to test for leak later in DHCP network

    In logs it looks like this:

    Code:
    Jan  1 04:06:21 guardian daemon.info xl2tpd[1809]: Using l2tp kernel support.
    Jan  1 04:06:21 guardian daemon.info xl2tpd[1810]: xl2tpd version xl2tpd-1.3.0 started on guardian PID:1810
    Jan  1 04:06:21 guardian daemon.info xl2tpd[1810]: Written by Mark Spencer, Copyright (C) 1998, Adtran, Inc.
    Jan  1 04:06:21 guardian daemon.info xl2tpd[1810]: Forked by Scott Balmos and David Stipp, (C) 2001
    Jan  1 04:06:21 guardian daemon.info xl2tpd[1810]: Inherited by Jeff McAdams, (C) 2002
    Jan  1 04:06:21 guardian daemon.info xl2tpd[1810]: Forked again by Xelerance (www.xelerance.com) (C) 2006
    Jan  1 04:06:21 guardian daemon.info xl2tpd[1810]: Listening on IP address 0.0.0.0, port 1701
    Jan  1 04:06:22 guardian user.info redial[1811]: Started. Time: 5
    Jan  1 04:06:22 guardian daemon.notice xl2tpd[1810]: Connecting to host tp.internet.beeline.ru, port 1701
    Jan  1 04:06:22 guardian daemon.notice xl2tpd[1810]: Connection established to 78.107.1.3, 1701.  Local: 61958, Remote: 8740 (ref=0/0).
    Jan  1 04:06:22 guardian daemon.notice xl2tpd[1810]: Calling on tunnel 61958
    Jan  1 04:06:22 guardian daemon.notice xl2tpd[1810]: Call established with 78.107.1.3, Local: 60293, Remote: 28739, Serial: 1 (ref=0/0)
    Jan  1 04:06:22 guardian daemon.info pppd[1812]: Plugin pppol2tp.so loaded.
    Jan  1 04:06:22 guardian daemon.notice pppd[1812]: pppd 2.4.5 started by root, uid 0
    Jan  1 04:06:22 guardian daemon.info pppd[1812]: Using interface ppp0
    Jan  1 04:06:22 guardian daemon.notice pppd[1812]: Connect: ppp0 <--> l2tp (78.107.1.3)
    Jan  1 04:06:28 guardian daemon.info xl2tpd[1810]: Can not find tunnel 15409 (refhim=0)
    Jan  1 04:06:28 guardian daemon.debug xl2tpd[1810]: network_thread: unable to find call or tunnel to handle packet.  call = 41665, tunnel = 15409 Dumping.
    Jan  1 04:06:28 guardian daemon.info xl2tpd[1810]: Can not find tunnel 1228 (refhim=0)
    Jan  1 04:06:28 guardian daemon.debug xl2tpd[1810]: network_thread: unable to find call or tunnel to handle packet.  call = 56537, tunnel = 1228 Dumping.
    Jan  1 04:06:33 guardian daemon.info xl2tpd[1810]: Can not find tunnel 1228 (refhim=0)
    Jan  1 04:06:33 guardian daemon.debug xl2tpd[1810]: network_thread: unable to find call or tunnel to handle packet.  call = 0, tunnel = 1228 Dumping.
    Jan  1 04:06:36 guardian daemon.info xl2tpd[1810]: Can not find tunnel 15409 (refhim=0)
    Jan  1 04:06:36 guardian daemon.debug xl2tpd[1810]: network_thread: unable to find call or tunnel to handle packet.  call = 41665, tunnel = 15409 Dumping.
    Jan  1 04:06:36 guardian daemon.info xl2tpd[1810]: Can not find tunnel 1228 (refhim=0)
    Jan  1 04:06:36 guardian daemon.debug xl2tpd[1810]: network_thread: unable to find call or tunnel to handle packet.  call = 56537, tunnel = 1228 Dumping.
    Jan  1 04:06:37 guardian user.info redial[1811]: WAN down. Reconnecting...
    It looks like l2tp handshake is fine, but no connectivity through l2tp tunnel.
    Sometimes it looks more quiet, without "Can not find tunnel" messages - it may be traces from previous connection attempts. Anyway, it hangs in ppp connection phase and after some hang time starts over and over.
     
  73. jyavenard

    jyavenard Network Guru Member

    I guess we have two problems now...

    For the leak, there's not much I can do without access to a device to quickly try. Especially if the bug is in the newer drivers.. I can try using the new drivers but with the older utilities..

    Did you ever try with the newer RT-N16 stock firmware? The one sharing the code with the rt-n66? Both for the l2tp issue and the ipv6 tunnel

    Would be interesting to
     
  74. guardian

    guardian Serious Server Member

    Well, I can simply connect the router to my network and give you http/ssh access, so you can set up an independent IPv6 tunnel from it (can even give you he.net credentials for that tunnel), but with this setup you will not be able to test l2tp.

    The other way -- we may probably connect through jabber/icq/msn/gtalk and agree on time/date, and at this time I would be ready to assist you in testing/flashing/rebooting/etc.

    Please let me know if you need this.
     
  75. jyavenard

    jyavenard Network Guru Member

    Well, for a start, I'd also like to fix L2TP connection for you, using the latest version of the l2tp drivers (as in use with the asus stock firmware)
    Merlin has made an image for the RT-N16 using the RT-N66 source code... that would at least give us some indication as to the issue is with the linux kernel used, or how things were backported to tomato

    Me be able to test the leak is something I'll be able to do by next week, while L2TP I'll never be able to test it myself unless I crash at your place :)
     
  76. guardian

    guardian Serious Server Member

    Well, ok, will try to install asuswrt-merlin RT-N16 latest build and will test l2tp/ipv6. Hope will have time for it this evening, around 18:00 GMT.
     
  77. guardian

    guardian Serious Server Member

    JYAvenard: thinking of setting up a real test lab for you with separate wan l2tp connection and RDP access to a host connected to the router. I probably can even setup a webcam to observe router leds :) The only problem is how to automate powering router on and off and remote control it... maybe some kind of intellectual UPS...

    To remotely connect to the lab, it would be enough to have relatively fast internet and some kind of RDP client (e.g. windows 7 pro/ultimate has it inside).
     
  78. jyavenard

    jyavenard Network Guru Member

    As your WAN is connected via L2TP, it's going to be a real struggle to test it. As, if it doesn't work, bye bye WAN
     
  79. guardian

    guardian Serious Server Member

    jyavenard: no, I'm talking about completely separate wan with its own ethernet and its own l2tp. It is possible for me to set up two separate WANs, one for my network (and your RDP connection), one for the router. The host you are connecting to should be multi-homed (it's simple - WiFi connection to base LAN, say, 192.168.2.0/24 and ethernet connection to testing router, 192.168.1.0/24), has attached webcam looking at the router leds, and has UPS control GUI software.

    It will cost me some money and some time, but I can absolutely afford it. The only question is: do you really want to spend your efforts and time using it :) Because if you wouldn't, all this is worthless to setup.

    In any way, I'm ready to follow your directions and make tests for you in a usual way :)
     
  80. jyavenard

    jyavenard Network Guru Member

    My RT-N66 just arrived.. so at least I can perform some tests myself now
     
  81. guardian

    guardian Serious Server Member

    jyavenard: have tested RT-N16_3.0.0.3_178.15.trx, which is latest asuswrt-merlin beta for Asus RT-N16.

    Results:
    - L2TP works, and works much faster than on Tomato (even faster than on RT-N66, which has more CPU power), it looks like nearly all my 100mbps connection is available over an L2TP tunnel, no CPU bottleneck limit. This is very surprising.
    - 6in4 tunnel works, no memory leak
    - DDNS to tunnelbroker.net is somewhat broken, but this is out of scope of this discussion

    Great results with L2TP on merlin's build. Hope this will somehow be ported to Tomato...
     
  82. Riddlah

    Riddlah Networkin' Nut Member

    loaded up Tomato Firmware 1.28.0501.10 MIPSR2-jya-shibby-RT-N K26 USB AIO (AIO with IPv6 patch + updated radvd) yesterday. When I checked last night I was sitting at roughly 107/108 mb free. Just logged in and checked, I am currently sitting at 85 mb free. I'll be keeping an eye on it and verify if it drops lower.

    [​IMG]
     
  83. jyavenard

    jyavenard Network Guru Member

    ok...

    so it does leak still even there...

    not surprising as there was nothing in the ipv6 patch that actually touch ipv6
     
  84. jyavenard

    jyavenard Network Guru Member

    By default, with no QoS, no port forwarding etc... The asus firmware uses hardware accelerated code, and it does fly.. Unfortunately, once you active QoS, port forwarding or other router functions they are disabled and then it works much slower. Tomato never have those running. When used fully, tomato compares very well with the asus one
     
  85. jyavenard

    jyavenard Network Guru Member

    I'm having little luck using IPv6 here, even using my ISP's provided /64.
    I get my IPv6 block from the ISP, on the router I can run ping6 anywhere just fine.

    However, none of my local machines can use it. The only thing I can ping over IPv6 is the router. I get the same result using either native IPv6 or Hurricane Electric tunnel.
    same result between either tomato and asus stock firmware...

    tad hard to test memory leaks if I get no traffic to start with
     
  86. USNetboy

    USNetboy Networkin' Nut Member

    Sounds like your hosts are missing routes to the router. Is radvd running correctly on your router? Can you see your router's link-local address setup as default route on the hosts?
     
  87. gingernut

    gingernut LI Guru Member

    FWIW Asus fixed the HW acceleration getting diabled once port forwarding was used in most recent builds, at least in RMerlin's releases, it was a bug.
     
  88. JugsteR

    JugsteR Serious Server Member

    I have read that tomato uses features that are "incompatible" with that hardware acceleration. But if that was just a bug then maybe it could find its way into tomato someday?

    Signature? Signature!
     
  89. guardian

    guardian Serious Server Member

    jyavenard: Would you like me to test something more for you?
     
  90. jyavenard

    jyavenard Network Guru Member

    I got it going with HE 6in4 tunnel now.
    No go with the ISP's native IPv6...
    Yes, the router's link-local address is set as default router on all the local clients...

    Well, I can start with that at least...
     
  91. jyavenard

    jyavenard Network Guru Member

    I don't think it was a bug, it was intentional... The source code clearly commented about disabling the ctf module when virtual server was enabled or QoS was on...
     
  92. gingernut

    gingernut LI Guru Member

    OK, so it must be a workaround RMerlin found.
     
  93. jyavenard

    jyavenard Network Guru Member

    Merlin doesn't modify much the asus firmware, such core functions is something Asus changed... Will need to look at the kernel differences.
     
  94. kthaddock

    kthaddock Network Guru Member

  95. jyavenard

    jyavenard Network Guru Member

  96. Dark_Shadow

    Dark_Shadow Addicted to LI Member

    Got A 6to4 IPv6 Tunnel setup through HE, So I can test too.
     
  97. pmd

    pmd Networkin' Nut Member

    You're using Internode aren't you? Using pretty much any version of TomatoUSB in the last 20 months, these settings work:

    DHCPv6 with Prefix Delegation
    Prefix Length: 56
    Enable Router Advertisements: On
    Accept RA from WAN: On or Off, doesn't matter.
    Accept RA from LAN: Off/Disabled

    Paul
     
  98. M_ars

    M_ars LI Guru Member

    Asus also had that IPv6 leak according to their changelog:

    Code:
    ASUS RT-N16 Firmware Version 3.0.0.3.162
    Fix bugs:
    1. UI related bugs
    2. Network map cannot show device name.
    3. QoS control for wired connected devices.
    4. Special character name/password issue in VPN server.
    5. Download master installing issue in AP mode.
    6. PPPoE compatibility issue.
    7. L2TP compatibility issue.
    8. Sonos audio system and network map compatibility issue. Special thanks to Merlin for his contribution.
    9. IPv6 6in4 memory leak
    10.IPTV multicast related issues
    11.FTP/Samba account change issue
    @RMerlin, do you have maybe a contact person at Asus, that maybe could help us?

    Or does someone else know someone from the AsusWRT-Team/Asus who is a fan of Tomato and maybe willing to help us :) ?

    The patch from shibby does not hurt as far as i can see, but i am not sure if its the solution for the leak. Nevertheless we could/should update the 2 (3) files [drivers/net/pppol2tp.c + pptp.c and ppp_generic.c (my additional suggestion)] as suggested by shibby, its a more nice/better code :)
     
  99. jyavenard

    jyavenard Network Guru Member

    I have identified the source of the leak... Can't work on it until this week-end... it is related to the newer driver, luckily not in the binary blob
     
    kthaddock likes this.
  100. jyavenard

    jyavenard Network Guru Member

    No.. I'm currently in France, using Free (www.free.fr). They provide both native IPv6 and 6to4
     

Share This Page