How to speed up your slow Internet Part 1

If like me you have been left out in the digital revolution that is the fibre broadband rollout, or have a lot of things going on connected to the internet all wanting updates then you will know this can suck your connection speed and pleasure out of the 21st century.

Whilst we can’t force BT, Virgin or the others into laying the cables and getting everyone connected to the high-speed digital highway (yet!!), we do have some ways in which we can make your broadband a little better, true some of these methods are not for everyone and they can be costly to set up but once they have been set up then the benefits will outweigh the costs in no time. Over the years I have worked in some very large environment’s that have over 100,000 servers using Unix, Linux and Windows as well as VMWare and other operating systems, connected to these servers were up to 300,000 client PC’s, the servers and client PC’s were spread throughout the world and connecting them quickly was a task, but these big companies use technologies to make better use of what they have. In this post I am going to show you how to use the technology of the big companies to better your own internet connection and make it work for you, obviously big companies have large budgets but fear not we can do this on a shoestring, for as little as £100 or as much as £3000 you can employ this technology.

So what are you going to show us?

Well in the next three blog posts I am going to show you three ways in which you can improve the efficiency of your internet connection. The three ways I am going to show you how to do this. I do not use some software that claims to make it more efficient (and usually doesn’t). What I am going to show you is how to do this using a piece of hardware, this hardware can usually run one or more services, but you will want to make it as small and as power-efficient as possible as you will need to leave this device running 24/7 or have the device turn on and turn off on a schedule, my personal preference for this would be a Net Top PC such as an Acer Revo or in my case a Mac Mini. The device will need to be hard-wired into your router using a high-quality cable. So what are these three things?

  • Proxy Server
  • Caching Server
  • WSUS

The WSUS will cost you a home server licence however some may benefit from this especially if you have 5 or more computers in your household. In regards to the caching server, this is really only for Apple devices and would benefit you if you have more than 5 Apple devices in your household.

The proxy server is where I will start in the blog post and explain what it is, what it does and how you can make it work for you.

So what is a proxy server?

A proxy server is a computer that acts as an intermediary between the user’s computer and the Internet. It allows client computers to make indirect network connections to other network services. If use proxy server, client computers will first connect to the proxy server, requesting some resources like web pages, games, videos, mp3, e-books, any other resources which are available from various servers over the Internet. As soon as getting such request, the proxy server will seek for the resources from the cache in its local hard disk. If the resources have been cached before, the proxy server will return them to the client computers. If not cached, it will connect to the relevant servers and request the resources on behalf of the client computers. Then it ‘caches’ resources from the remote servers, and returns subsequent requests for the same content directly.

Nowadays, we use a proxy server for various purpose like sharing Internet connections on a local area network, hide our IP address, implement Internet access control, and so on. Below are some benefits why people use a proxy server:

  • To share an Internet connection on a LAN. Some small businesses and families have multiple computers but with only one Internet connection, they can share Internet connection for other computers on the LAN with a proxy server.
  • To speed up Internet surfing. If use proxy server, all requests from client computers will reach the proxy server at first, if the proxy server has cached the required resources in its local hard disk before with the web cache function, clients will get feedback directly from the proxy server, it will be more quickly than direct accessing.
  • To hide the IP address of the client computer so that it can surf anonymous, this is mostly for security reasons. A proxy server can act as an intermediary between the user’s computer and the Internet to prevent an attack and unexpected access.
  • To implement Internet access control like authentication for Internet connection, bandwidth control, online time control, Internet web filter and content filter etc.

So as you can see using this technology for more than 7 days will vastly improve your internet connection speed, as the proxy keeps a copy of the page and will only go out to the internet to fetch new data or if the page is not part of the cache, this is also useful if you are on a capped internet like myself when using the satellite broadband because the requests are for updates, not the full page, if you are like me using satellite broadband you will know how bad it can be with the ping ratio and this helps with that a little, as well as the page, is coming locally not from the internet, whilst you are getting the cached page the proxy is on the job of getting the updates so it’s a win all round

So you like this idea and want one for yourself

OK so you want one, you can see the benefits to having one of these in your household or small business, well you have two options for this, you can buy all the equipment yourself and if you feel confident enough with technology you can follow the below steps OR from as low as £250 + VAT you can buy one that is tailor-made to your requirements by us at Ghouletech Ltd, this will be set up and configured in your home and up to 4 PC’s connect to it as well as a quick tutorial on how it works and how to setup your new devices. Ok marketing out of the way (sorry about that) let’s get on with the tutorial to set up your very own proxy server. To begin with, you will need to asses your own home network and judge the following for yourself

  • How many devices do I have connected to the internet
  • What are the devices (Desktops, Laptops, Smartphones, tablets, TV’s and DVD players)
  • What is the primary OS (Windows, Linux or Macintosh (Apple)
  • How do I connect to the internet currently (wired or wireless) how many free wired network ports do I have (if the answer to this is none then you will require a switch)
  • How much internet do I currently use and what does my internet usage look like?

OK, so you have answered the above questions and have decided that you have a sub 2Mb line and have around 5 devices and it looks like you will need a proxy server to make your connection better. The next stage of this is to assess your knowledge of IT and where your comfort zone is, if your primary Operating System is windows then you may want to use a windows based proxy, if you have Macs or Linux devices you may want to build your proxy server as either one of those, if you have a good mix of Windows and Apple devices then you may want to go down the Linux route I have listed the pro’s and cons of each type of proxy below

  • LINUX Proxy
    • Pro’s
      • TCP/IP Stack more efficient
      • More stable OS
      • Benefits from community and Enterprise Updates
      • FREE OS and Proxy software
      • Will work on any old computer (even if it previously had windows on it)
    • Con’s
      • Requires technical knowledge
      • OS is not for beginners
      • Can require access to the command line (terminal) for setup and administration
    • Mac Proxy
      • Pro’s
        • Built on BSD (Unix)
        • Very Stable OS
        • Regular updates from Apple
        • Easy for novice users
        • Backed by Apples famous customer support for hardware and OS
      • Con’s
        • Expensive hardware (£399 for the base model)
        • Proxy software is not Apple
        • Can be a waste of resources if not using the server App (£15 extra)
        • If using other services such as Apple server then this can become complex
      • Windows Proxy
        • Pro’s
          • User-friendly OS
          • Can be used by anyone who has bought a PC in the last 30 years
          • You probably have an old unloved one somewhere in your house
          • Updates provided by Microsoft for windows
          • You can use the PC for other things
        • Con’s
          • TCP/IP stack is bloated
          • Very intensive on requirements to run
          • OS is not as stable as others
          • Requires more work to secure the OS and applications
          • Will require a firewall and Antivirus
          • Proxy Application will need to be from another company either free or paid for.

*Note: – the TCP/IP stack is basically the way in which the computer connects and talks to your local network as well as the internet (sorry for the technobabble).

Ok so you have now weighed up the options of the proxy and now know which proxy style is the best for you, later in this document I will link to a couple of websites that provide the best proxy software for the OS you are using, I am going to show you how to set up a proxy server in Linux as well as how to do it on a Synology NAS (the way in which I have it setup here). For most users, you will need the following pieces of equipment

  • Old PC or dedicated proxy PC this can be new if you want
  • OS License if its windows, Mac’s come with OSX and Linux is free to install on whatever you have laying about
  • Proxy software application downloaded

I remember the first proxy server that I built in the late 90’s I used a second hand Dx486 100Mhz that had a whopping 16Megs of RAM and a whole 2Gb hard drive, all but the hard drive was second hand, I splashed out on the HDD I remember using coral Linux and a very early version of Squid proxy server, it was noisy and would get hot and shut down every so often due the overheating but it worked my little 56K modem was connected and was told to dial on-demand (I was lucky to have two phone lines) the internet provider was Netscape Online using the free number 🙂

How to install Squid on a Synology

This is probably one of the easiest installs that you will ever do, Synology now has a proxy service that you can download and configure in its GUI, and you can do this if you wish but the more capable Squid is a little more work and you will need to turn on SSH and connect to the NAS as root through SSH (using Putty or terminal).

The old way

In my example, my NAS IP is 172.16.100.100 and my IP range on my LAN is 172.16.0.0 – adjust this accordingly below:

  1. Install squid: ipkg install squid
  2. Adjust Squid’s config-file located in /opt/etc/squid/squid.conf:
  3. Take note from my above config, that I chose a cache-size of 20(!) GB (cache_dir).
  4. Validate your Squid configuration with squid -k parse
  5. Create the Squid cache-directories with squid -z
  6. Start Squid manually to check for errors: squid -NCd1
  7. Create a symbolic link so that Squid starts automatically: ln -s /opt/etc/init.d/S80squid /usr/syno/etc/rc.d/
  8. Once you restart the NAS, Squid should be started automatically (log files are in /opt/var/squid/logs)

**Dummy error: Happened to me – if Squid starts and you don’t notice any improvements in browsing speed, make sure that you have your browser’s proxy settings adjusted. I will show you how you do this later in this post, or if you have bought a proxy solution from me I will show you this in a one to one session.

How to install on Ubuntu

Ubuntu is a Linux variant based on Debian, it’s a relative newcomer to the Linux scene and is distinct in the fact it has become very user-friendly to use and install. Being Linux most of the software that you will use on this platform is free to use, again because Squid is open source I will go over the above again but installing under the Ubuntu way

  1. Install squid: sudo apt-get install squid3
  2. Adjust Squid’s config-file located using vi /etc/squid3/squid.conf
    http_access allow local_net

acl local_net src 192.168.0.0/255.255.255.0

  1. Validate your Squid configuration with squid -k parse
  2. Create the Squid cache-directories with squid -z
  3. Start Squid manually to check for errors: squid -NCd1

Ok so now you have your Squid Server running either on a NAS device or an old PC connected to your network, you need to know a little about the logs and have some idea whats going on if something isn’t working for you, you should familiarise yourself with the statuses in Squid’s access-log:

  • TCP_HIT: A valid copy of the requested object was in the cache.
  • TCP_MEM_HIT: A valid copy of the requested object was in the cache, AND it was in memory so it did not have to be read from disk.
  • TCP_NEGATIVE_HIT: The request was for a negatively-cached object. Negative-caching refers to caching certain types of errors, such as “404 Not Found.” The amount of time these errors are cached is controlled with the negative_ttl configuration parameter.
  • TCP_MISS: The requested object was not in the cache.
  • TCP_REFRESH_HIT: The object was in the cache, but STALE. An If-Modified-Since request was made and a “304 Not Modified” reply was received.
  • TCP_REF_FAIL_HIT: The object was in the cache, but STALE. The request to validate the object failed, so the old (stale) object was returned.
  • TCP_REFRESH_MISS: The object was in the cache, but STALE. An If-Modified-Since request was made and the reply contained new content.
  • TCP_CLIENT_REFRESH: The client issued a request with the “no-cache” pragma.
  • TCP_IMS_HIT: The client issued an If-Modified-Since request and the object was in the cache and still fresh.

Once you have all this information your good to go with connecting your client PC’s to the Squid proxy server.

Connecting your Client PC’s to the proxy

Ok so your all up and running with the server-side of things on your network, you are going to want to let your other PC’s and devices know that its on the network and that it is here to help, below I have set out how to connect to the proxy using different browsers, if you have smart devices (phones, TV’s or DVD players) then you are going to need to consult your manual, I will focus on Internet Explorer and Firefox as these are the main contenders for web browsing however most other browsers follow the same principles you just have to find the proxy tab in the settings (even most chat programs such a skype and MSN have proxy settings that you can set.

Use a Proxy Server for IE

Click “Tools” -> “Internet Options” -> “Connections” -> “LAN Settings” -> select “Use a proxy server for your LAN” -> “Advanced”, configure as bellow.

Internet Explorer Proxy

Use a Proxy Server for Firefox

Click “Tools” -> “Options” -> “Advanced” -> “Network” -> “Connections” -> “Settings” -> “Manual proxy configuration”, configure as bellow.

Firefox Proxy

Conclusion

Ok so your done, you now know what a proxy server is and how you can set one up and configure the rest of your PC’s to connect, hopefully you should see a 200% increase in the efficiency of your internet connection as well as up to a 30-40% decrease in the bandwidth usage, whilst this won’t help with your online first-person shooters or fix all of your internet woes, it will go a long way to reduce the time waiting for a page to load. Stay with us in Part two I will show you how to set up a Caching server for all Apple software updates so you can download all the apps and updates automatically to one device and distribute to your other Apple devices without having to download all over again, and in Part 3 I will show you how to set up a WSUS server (windows updates) using Windows home server software.

As you can see there is no silver bullet to this but with a little work, we can make things better for ourselves. for more information on proxy server software please see here, most of the links will trackback to the original software providers webpage and should have help files to help you set one up yourself.