Fiddler: Trying to work out how it all hooks together
I mentioned previously that we're making use of Fiddler quite a lot on my current project, mainly to check the traffic going to and from the service layer, and I'm quite curious how it actually works.
In particular I wanted to know:
- How we're able to route requests through Fiddler and then through the corporate proxy
- How proxy settings work differently for Firefox and Internet Explorer
As far as I'm aware the source code for Fiddler isn't available so a colleague and I tracked the various proxy settings when Fiddler was turned on and off and also had a look at some registry settings.
As Internet Explorer is a WinInet based application it looks at the Local Area Network settings to check whether there are any proxies that it needs to route requests through. I believe any proxy defined in here would be known as the 'system proxy'.
We noticed that our application also routed its requests via the proxy defined in the LAN settings as well – it is a .NET application but I guess that even applications written in other languages but running on windows would make use of the system proxy too?
Firefox has its own proxy settings which are accessible via the preferences menu of the browser and are then configurable from the 'Advanced -> Network' tab.
If another proxy is defined in the LAN settings when Fiddler starts up it takes this value and stores it somewhere (in memory presumably) as the 'Upstream gateway proxy' (Tools -> Fiddler Options -> Connections) which it directs requests through before they go out to the internet.
It then changes the proxy defined in the LAN settings to point to itself – typically port 8888 locally.
The 'Upstream gateway proxy' always seems to be the same as the 'system proxy' so if a different proxy is defined on FireFox than Internet Explorer this won't be used even though on the latest version of Fiddler when it starts up the FireFox proxy will be changed to point to Fiddler.
When Fiddler is shut down it reverts the proxy settings to how they were previously although (unsurprisingly) if it crashes then we are left in a state where requests are being sent to 'http://localhost:8888′ and there is no application listening on that port anymore meaning that we end up with a 'Cannot find server or DNS Error'.
Restarting Fiddler gets rid of that problem although the 'Upstream gateway proxy' usually seems to get lost so if we want to make requests out to the internet then we need to go and reset that in our LAN settings before restarting Fiddler.
If there's anything inaccurate in what I've written or if you know of any resources that would help me understand this better if you let me know in the comments that'd be cool!
I can't find the reference just at the moment, but:
Whether WinINet proxy settings affect you depends on which low-level API is used to set up the socket that the http request will be made over. All of the high-level .net libraries end up using low-level calls that are affected by WinINet.
I think the alternative API is only usable from direct win32 code, or maybe would require rewriting an http handling library in .net.
Dave Cameron
7 Sep 09 at 4:43 am
For the record, a lot of Windows applications will use what you call the "system proxy" settings. Safari and Chrome, but not Eclipse. The Cygwin installer has it as an option called "Use IE5 settings".
One thing that mightn't be obvious (although it's probably not relevant here) is "automatic proxy configuration". Proxy autoconfig (PAC) isn't just a hostname/port pair, it's actually a script. That means that any client that wants to use those settings needs a Javascript interpreter somewhere along the way.
There's also the Web Proxy Auto-Discovery protocol, which uses DNS to find a PAC script. (Theoretically it uses DHCP first but I don't think most implementations use that.)
Rob Hunter
7 Sep 09 at 10:48 am
Hi Mark,
What did you use to draw the pictures on this post…I like them a lot.
Cheers
Mat
mat roberts
11 Nov 09 at 12:29 am
@mat – I used yuml – http://yuml.me/
Mark Needham
11 Nov 09 at 12:36 am