FAP Upload Meter Gone Crazy?

[luomat]

On 12/01 my "Actual Usage Upload" for the past 30 days was 2,343 MB.

Today (barely 12/04) it's 858 MB.

Am I seriously supposed to believe that I **unknowingly** uploaded almost 1500mb in two days sometime last month?

Or is it more likely that something went wrong with their FAP meter?

(I'm running an all Mac household, so there's no spyware-ridden zombie Windows machine on my network, and the wireless is locked down with WPA2, so there's no one else piggybacking on my connection. And there's no P2P usage here.)

[c3ajeff]

I'm on the Riverside beam which went down Wednesday evening for maintenance. Before the maintenance shutdown, My D/L meter showed just over 5GB... within 5 minutes of it coming back up again I checked and it was over 7.5GB!!! Now tell me how I can download over 2.5GB when I can't even get an IP from WB???

They never sent a warning, just an FAP violation email on Thrusday morning. Uh huh. Then early this morning, another email telling me the FAP violation was over... then this evening another FAP violation letter... You would think these guys could make up their minds.

Something is wrong, very wrong. I have a byte counter (which I set up after the only other FAP violation I had, which didn't add up either) which measures ALL TCP traffic on my ENTIRE network (3 computers in total) and the 30 day rolling total is well over 30% less than what WB says I am downloading. It's just NOT POSSIBLE but do you think they listen? HA.

They don't guarantee service... maybe I shouldn't guarantee payment? Yeah right. I think this is all an effort to try to get me to upgrade and pay them more $$$


Jeff

[Random]

Wow your not the only one, I went to check my fap just a few mins ago, ( hadnt checked in a few days, and boom 3-5 GB D/L, Yellow Bar and you are approaching your FAP threshold! ahh no i have even downloaded that much ever!

[JoshS]

Hi,

I've been with WB since May and have been pleased with the service, for
the most part.

I joined the forum over a week ago and my acct was just approved, so this info below is over a week old...

Back in October, we had several days of high downloads...nearly went
over our limit, but didn't. Since then, we've been watching our usage,
and just in the last 3 days has it fallen back to normal. About two weeks ago I installed TrafficStatistic on all of our computers, so I could more
accurately moniter our usage. I also have Fapzilla recording data every
hour.

On Dec 5, I logged on to the WB Tech Chat and questioned the guy about
my usage. I asked him to email me my daily stats from the last 30
days...which he did. After looking the stats over for a few minutes, a
couple things just weren't adding up.

1. At 10:15 am, the stats he sent me said we'd already used 72mb. We'd
only been using the internet since 8:30am...no way could we have used
that much already. The TrafficStatistics from all computers added up
showed we used just 32mb.

2. The stats he sent showed we had downloaded 727mb on Nov. 30. That
sounded extreme.

I logged back into chat to question him about these things. Then things
started getting interesting...

1. He claims that the slower my connection, the higher my bandwidth
usage will be. Multiple computers and using a router will slow things
down, resulting in higher usage.

2. I said my TrafficStatistic stats for the morning were way below what
the stats he sent me showed. He claimed TrafficStatistic only measured
the traffic on my LAN and didn't account for the traffic from my modem
to the satellite. In effect, I'm only counting half of my bandwidth
usage. He gave examples that made no sense.

3. Finally, I asked him to tell me my usage for the day, again. I was
curious to see how much it changed in the hour that passed since he
emailed me the first set of stats. Suddenly it had cycled through for
the day (whatever that means!) and he couldn't tell me.

Then he said he had helped all he could and told me to call Tech support.

Is anything he told me true? I'm skeptical, but maybe things don't work like I thought.

Also, has anyone experienced bandwidth that won't go down?

If I'm not mistaken, TrafficStatistic will measure ALL traffic even from trojans, etc...right?

I'll attach my chat transcripts, if anyone cares to read exactly what was said.

I appreciate your thoughts!

[lhookins]

I don't know what to make of your monitor showing only half the actual usage. That does seem odd.

The fact that he couldn't give you usage for that day was probably because his monitor software doesn't update frequently, possibly only once a day. In that case, he could give you usage from previous days, but not the usage for that day.

The only thing I can imagine that would cause a slower connection to use more bandwidth for a file transfer is that it would send and receive smaller packets, and each packet has some overhead. It would basically take the same amount of overhead for each packet, so if there are more packets (because each one was smaller) you'd get more overhead. Not nearly enough to double your usage though.

Good luck, and hang in there.

Hook

[Spice300]

As to the slower connection thing, if there are errors in the packets the modem could drop the packets and request that new ones be sent. Since the modem is requesting resent packets, your computer would not see them.

In my experience WB usually over states my usage when I do a lot of browsing to different sites. Sometimes I cancel a large page load before it completes. There is some speculation whether WB's FAP meter counts the amount of data sent through the satellite or the amount that arrives at the gateway. WB's proxy or Sky-X gateways probably prefetch the entire page and the FAP meter counts all of it. If the user cancels the page load, then he receives only a portion of the page even though the FAP meter includes the whole thing. I am not saying this is fair. I am giving a possible explanation for the discrepancy based on my experience which is posted in the FAP section of this site. WB usually overstates my usage by 8%. Here is my most recent FAP graph showing an 11% overstatement.

[SMKolins]

I can see several ways that FAP might go above the actual download....

a) retransmission of packets
b) inefficient communication
c) total site preloading

a) has to do with a variety of causes that make individual packets get re-transmitted because they got lost or garbled along the way. A machine will receive a train of packets in a coded order and if a packet is lost your machine asks for that packet again (so that adds to the FAP count) and then you get the repeated packet (which again adds to your FAP count) assuming it takes just one re-transmission request to get your lost/garbled packet sent again. You can have retransmissions of retransmissions. To the end user it looks like bandwidth goes way down because end machines ignore retransmissions for end user counts - the retransmissions are considered a kind of "need to know" layer of complexity. It's possible WB is counting retransmissions against us. Tweaking advanced network settings could reduce retransmissions but in a complex network with many computers they will step on eachother toes somewhat - the router "settles" some disputes by dropping packets and letting the rules of TCP/IP recapture lost packets. A simple stream up and down from one machine will have fewer retransmissions and optimizing one machine will be better than trying to optimize many machines not to mention their shifting demands for throughput - but that may be possible too....

b) was just mentioned. The idea here is that you can have packet sizes that are being fragmented because they are too big or packets can be part of a stream of communication. For individual packets if a link in communication can't handle 1500 byte packets then the break a 1500 byte packet into two smaller packets and you get each returned with 1500 bytes but only a fraction of each is carrying data. That doesn't seem to be the case - WB itself seems to use 1500 byte packets and most places on the internet do to (but if you are pulling from someone who is on a PPoE link they have smaller packets they can send.) But there's more than packet sizes - Imagine trying to have a conversation across a canyon. There's a significant delay speaking and being heard as well as hearing the reply. So if you have a lot to say it might be good to let a whole sentence go- just enough as to fill the gap with a continuous stream of verbiage so that an acknowledgement is of a full burst rather than one word at a time (imagine someone nodding verbally to what you are saying to give you feedback that you've been heard - "uhuh"... "uhuh"....) If the acknowledgement has to happen as it would on an ethernet network then there will be many acknowledgements with few packets in the mean time. You could cut the number of acknowledgements if you wait for the stream of data that could be in flight to finish - fewer "uhuhs". Mind that acknowledgements are also the point where retransmissions are asked for - "wait, I missed that - say again?". Mind the cost of lost traffic goes way up if the whole sentence has to be repeated....

c) there's been some discussion that the WB optimizations include streaming to your WB modem a load of all of a webpage you are looking at as soon as it knows your are trying to see some of it. Thus imagine try to imagine downloading one file from a website of many large files. You are perhaps trying to download just the one file but the whole webpage it's part of is being sent to your WB modem in anticipation of your seeing them - even if you hit stop on the page load and started just downloading the one file. But the whole page is counted against your FAP. One way out of this might be to use a filtering proxy and not using the WB proxy suggestions so you don't get the preload and also don't load all of a webpage (avoiding adverts...)

I think all these points have been discussed in one or another thread but I haven't seen a summary post anywhere. Perhaps part of the problem is the shifting sands of configuration as WB changes it's operating norms or it could be that shifting configurations anywhere change what it means to be optimized and optimizations are notorious for causing extra errors if the situation changes and specific optimizations are now in fact wrong. So you spend many hrs getting it right but some clouds come in the way and suddenly all the optimizations are wrong.

I don't know. I'm just getting into figuring this all out.

[Spice300]

Originally Posted by JoshS

I'll attach my chat transcripts, if anyone cares to read exactly what was said.

I appreciate your thoughts!

I think the CS rep., Donovan, is mistaken when he claims that the counter is located in the customer's modem. I think the FAP counter is located in the gateway. I have had some days in which WB's FAP meter has read twice the usage reported by netstat, but it is usually related to browsing and canceled page loads. Here are some reasonable explanations that have been advanced to explain why WB's FAP meter over reports usage compared to monitors running on our computers:

1. millions of bytes verses megabytes. Wildblue's download threshold of 7,500 MB for the Value Pack might be in units of "millions of bytes," not megabytes (1,048,576). If your actual usage is measured in megabytes, this would cause WB's FAP meter to always read about 5% higher. See my linear error graph and the discussion that follows it in the thread.

2. satellite wrapper. Supposedly the SkyX gateways put a block of data into a packet to be sent through the satellite. There is some header information in the packet that is stripped off by the modem and never counted by your computer.

3. canceled page loads. Your FAP meter located at the gateway might count the entire page even though your computer received only a portion of it.

4. end of month flat line. Since January 2007, the FAP meter stops recording usage on the 30th and 31st of the month.

5. Inconsistent delays in updating the FAP meter. For the last several months my FAP meter has been updating quickly, in less than an hour so this may not be much of a factor anymore.

6. FAPPeriod, the time interval for subtracting usage, is not 30 days. It varies based on the number of days in the prior month. For example, because November had 30 days, FAPPeriod is currently 29.8 days. Last month, because October had 31 days, FAPPeriod was 30.8 days.

[JoshS]

Interesting info everyone...thanks!

It does sound like we are probably getting shafted...no matter how you look at it. I'm happy to report my usage is back to normal now, though.

So...does anyone have any good tweaks to minimize bandwidth? If we are docked for an entire page, even if we don't let it load...does that mean blocking ads and scripts really doesn't help, either?

[Spice300]

Originally Posted by JoshS

If we are docked for an entire page, even if we don't let it load...does that mean blocking ads and scripts really doesn't help, either?

I think so but have not tested it carefully. The theory is that the ad will be download to the modem and count against the FAP. Because Firefox's Ad Blocker Plus never requests it, the ad is not transfered from the modem to the computer. The ad blocker gets rid of the flashing lights and annoying banners and speeds up the page load but it probably does not reduce the FAP.

If this theory is true, then ad blocking software may be another cause of the FAP meter reading higher than the actual usage.

If such a device exists, an ad blocking proxy used in place of WB's proxy would block ads before entering the gateway, and thereby reduce the usage on the FAP meter. Be wary, though, because the term, "ad blocking proxy," is often used to describe a piece of software installed on the computer whereas I am referring to a hardware proxy connected to the Internet prior to a WB gateway. If anyone has an IP of an ad blocking proxy, I would like to try it.

[grover92000]

Originally Posted by Spice300

I think so but have not tested it carefully. The theory is that the ad will be download to the modem and count against the FAP. Because Firefox's Ad Blocker Plus never requests it, the ad is not transfered from the modem to the computer. The ad blocker gets rid of the flashing lights and annoying banners and speeds up the page load but it probably does not reduce the FAP.

I did some testing of this last year. I maintain some servers with one of my contracts so I have access to the web logs. I tested with an old version of Netscape and a modified fapzilla script. With Netscape I disabled everything - javascript, image loading, etc. I pulled the main site page. I saw the initial request come in followed by a blast of requests for the images, flash, etc a second later. With the modified script I only pulled in the main index.html. The result was the same. In both cases every object on the page was requested from my IP even though I know they were never requested by my system. As to whether or not they count against the FAP is anybodies guess since we don't know where the numbers are recorded. My reported vs. actual FAP download has always 10-20% higher than it should. I would say that, in my case at least, everything on a page counts against the FAP.

[SMKolins]

Originally Posted by grover92000

I would say that, in my case at least, everything on a page counts against the FAP.

And if you don't use the WB proxy pac file?

[grover92000]

Originally Posted by SMKolins

And if you don't use the WB proxy pac file?

I don't. And I have all the windows PC's here going through my own squid caching proxy.