Code Blog

Displaying 26-30 of 46 results.
2016/04/03 Network

Linux comes with a built-in firewall called IPTables. It has 3 default chains - INPUT, OUTPUT and FORWARD.

Input chain Coming from outside the firewall to destination inside the firewall
Output chain From inside the firewall going to outside the firewall
Forward chain Routing through the firewall (coming from the outside the firewall passing on to outside the firewall) 

 IPTables rules are normally put in a script file and run during start-up. Each chain will need to have a default policy (set with -P, see example below). The default policy for a chain can be either DROP or ACCEPT. 

Usage Examples:

iptables -P INPUT DROP // Set policy for chain to deny connections (drop packets)

iptables -A INPUT -p tcp --dport 8080 -j ACCEPT   // Add rule to end of chain: Accept incoming tcp connections at port 8080

iptables -A INPUT -j SHADOWSOCKS // Add to end of chain: Jump from Input chain to Shadowssocks chain (and will automatically go back to Input chain after finishing Shadowsocks rules)  

 Nat table (network address translation)

 iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-ports 8080 // Redirect incoming tcp traffic from port 80 to port 8080

 iptables -I INPUT -j SHADOWSOCKS // Insert at beginning of Input chain

// Count traffic coming in/out from

iptables -A FORWARD -s

iptables -A FORWARD -d 

iptables -nvx -L FORWARD // Print traffic counters

iptables -Z  // Clear all counters (for all rules)

iptables -Z FORWARD // Clear counters for Forward rule

iptables -R FORWARD 1 -s // Replace rule nr 1 in Forward chain with same rule. This will reset counter

iptables -I FORWARD -i eth0 -j TRAFFIC_ACCT_IN // incoming interface eth0

iptables -I FORWARD -o eth0 -j TRAFFIC_ACCT_OUT // outgoing interface eth0

iptables -vnxL FORWARD | awk '{print $2}' // print the 2nd column

iptables -vnxL FORWARD | awk '/delegate_forward/ {print $2}' // Print 2nd word in line that contains 'delegate_forward' text

The important rules regarding NAT are found in the 'nat'-table. This table has three predefinded chains: PREROUTING, OUTPUT und POSTROUTING.


// How is Shadowsocks set-up? I want to count the total traffic going through the router, but I also want to count the amount of data sent through Shadowsocks tunnel. The Shadowsocks tunnel is listening at port 1080. 

iptables -N SHADOWSOCKS // Create new chain with name "Shadowsocks"


iptables -A SHADOWSOCKS -p tcp // Count all incoming traffic

iptables -A SHADOWSOCKS -p tcp -dport 1080 // Does this work?

iptables -L SHADOWSOCKS -n -v -x // Display traffic data for Shadowsocks chain

The -x flag means you will get the exact value of the packet and byte counters, instead of only the rounded number in K’s (multiples of 1000) M’s (multiples of 1000K) or G’s (multiples of 1000M). 


2016/03/30 VPN,OpenWRT

Useful commands from ssh terminal

/etc/init.d/shadowsocks enable
/etc/init.d/shadowsocks start
/etc/init.d/ChinaDNS enable
/etc/init.d/ChinaDNS start
killall dnsmasq 
/etc/init.d/dnsmasq start
iptables -L -n -v
iptables -L OUTPUT -n -v iptables -L FORWARD -n -v cat /proc/meminfo cat /proc/cpuinfo netstat -tulpn

Note: dropbear is the ssh-server included in OpenWRT. Its a lightweight ssh-server.

shadowsocks-libev vs shadowsocks-libev-spec

shadowsocks-libev includes ss-{local,redir,tunnel}. Default config is running ss-local creating a local SOCKS proxy. shadowsocks-libev-spec is a special version for OpenWrt, that includes ss-{redir,rules,tunnel}. ss-redir will create the transparent proxy. ss-rules generates the proxy rules. ss-tunnel provide UDP transmission. Starting from v1.5.2 it uses the LuCI interface.


PandoraBox is a router firmware fork of OpenWRT, made by the Chinese OpenWRT community. The Intellectual Property and copyright laws must be different in China, since the source code contain Mediatek copyrighted material, which is clearly not allowed in the standard version of OpenWRT. Basically PandoraBox added wifi-support for routers still not supported in OpenWRT, but the source code cannot be added to the OpenWRT distribution for copyright reasons. If OpenWRT lack support for a router it is possible PandoraBox may support it instead.

GFWList - Sites blocked in China

We have already set up the China accelerated list in my previous OpenWRT post. However there is also the foreign_list.conf (GFWList) that can be added to dnsmasq.

In this way the logic will be:

  1. Check China accelerated list: Directly resolve IP if match is found, otherwise continue
  2. Check China blocked list: Directly resolve IP if match is found, otherwise continue 
  3. Use ChinaDNS for all remaining DNS requests. ChinaDNS will decide if it will use China DNS server or tunnel to DNS server outside of China. ChinaDNS uses the /etc/chinadns_chnroute.txt file to determine if an IP is in China or not.

Traffic statistics


vnStati - provides image output support for statistics collected using vnstat


Good list of different software for bandwidth monitoring:

Update: vnStat worked but did not have a lot of options to configure. I wanted to see amount of traffic going through Shadowsocks tunnel but this didnt seem possible with vnStat.

OpenWRT Ad-blocker - Privoxy

The privoxy package is designed to filter out traffic going to known ad-site servers. I could not find a package build in the stable Chaos Chamber repository, but there is a package build available in the snapshot/trunk repository, see:

Snapshots are automatically built every 1-2 days from the SVN trunk (development) repository sources by the buildbot. They are untested and might not work properly.

opkg update
opkg list | grep privoxy
opkg install privoxy
opkg install luci-app-privoxy

Update: Seems I had some problem to get this package to work well with Redsocks2/Shadowsocks/ChinaDNS so I removed privoxy and vnstat and the router was working well again. To remove packages from OpenWRT run "opkg remove packagename".

Alternative to ChinaDNS - DNSCrypt

DNSCrypt uses strong encryption for DNS traffic.There's an updated list of public servers DNSCrypt can use. For more information see 

For OpenWRT one need to install the package "dnscrypt-proxy".

/etc/dnsmasq.conf configuration file example: 

# Ignore the resolve file /etc/resolv.conf
# Use DNSCrypt as upstream DNS server 
# Turn off DHCP for the network interface 
# Local hosts file 

Alternative to Shadowsocks - obfsproxy

Shadowsocks may not be enough because of deep packet inspection used by the GFW. One alternative is using an obfuscated SSH tunnel, based on Tor's obfsproxy.

DNS cache: pdnsd

pdnsd is a DNS server designed for local caching of DNS information. Correctly configured, it can significantly increase browsing speed on a broadband connection. Compared to BIND or dnsmasq it can remember its cache after a reboot; "p" stands for persistent.

I can't find a package to install for my version of OpenWRT, might need to build from source code.

dnsmasq -> ChinaDNS -> If China: / If outside China -> pdnsd -> ss-tunnel ->

Other notes & links to websites

There is a shadowsocks version using polarssl, so I quick check online revealed the following:

mbed TLS (previously PolarSSL) is an implementation of the TLS and SSL protocols and the respective cryptographic algorithms and support code required.
Its a light-weight open source cryptographic and SSL/TLS library written in C with small memory footprint.

Then there is another package available called ShadowVPN, which seems to offer similiar functionality as Shadowsocks+Redsocks2 but maybe faster and more lightweight, see link:
ShadowVPN is a fast, safe VPN based on libsodium 

Both ShadowVPN and PolarSSL might be interesting to test out. I'm not really sure how to set up a ShadowVPN server that the OpenWRT can tunnel to though but maybe there is more information available if I would actually install the package.  

I found a lot of good and helpful information (in Chinese) at  
For example the tester script I have copied from

Now here's a story, the guy that developed shadowsocks (clowwindy) is Chinese, and run into some trouble last year:

On August 22 2015, an open source project called ShadowSocks was removed from GitHub. According to the project’s author, the police contacted him and asked him to stop working on the tool and to remove all of the code from GitHub. He later removed the reference of the police, presumably under the pressure of the police.

I guess he mentioned the police in the first readme notice as the source code was taken down. Now the "" file at github reads "Removed according to regulations.". That's just insane. China plays by other rules.

More tips

Speed up Apple downloads:

Create the file /etc/dnsmasq.d/apple.conf and add the line below to it:

I read online that:

"shadowsocks-libev has ss-redir which works as a transparent proxy, so redsocks is not needed"

Not sure about this, will need to try it myself


2016/03/30 MacOSX

Before switching to Mac I used to install Cygwin ( to add some handy Unix command line tools to Windows. As Mac OS X is built from Unix it already comes with a lot of command line tools pre-installed. However, Homebrew ( is a package manager for installing some Unix tools that are missing in Mac OS X. It's also quite interesting that Homebrew used Kickstarter to fund part of its setup costs, see extract below from Wikipedia:

In March 2013, Homebrew successfully completed a Kickstarter campaign to raise funds for servers to test and build formulae and managed to raise £14,859.

Links  Introduction to Homebrew Homebrew Cask extends Homebrew and brings its elegance, simplicity, and speed to OS X applications and large binaries alike. is an online package browser for Homebrew Online search for Homebrew packages

Homebrew typically deals with command line software, normally distributed under an open source licence. brew cask is an extension to brew that allows management of graphical applications through the Cask project. Cask software may have commercial licences.

Useful HomeBrew Packages

wget Command line network downloader

Network exploration tool and security / port scanner

youtube-dl Download Youtube videos
ffmpeg Encode/compress video & music
htop Improved top tool; Scroll process list etc
imagemagick Tool for rendering image files
ack similiar to grep (text finder)
tig git command line interface
awscli AWS command line tools (Amazon hosting)
geoip check where and IP comes from
ccrypt file encryptor
hh shell history browser / searcher
mysql database system
automysqlbackup automates MySQL backups
irssi IRC client

Homebrew are by default installed in /usr/local

To install a package just type "brew install packagename".

I also tried to install "octave" from Homebrew (a free GNU Matlab-clone) but it's GUI is unstable for Mac OS X so I didn't bother too try to get it working in the end (Scilab, another Matlab clone, may be better, see

Youttube-dl commands

youtube-dl -F list all formats
youtube-dl -f 22 download format number 22
youtube-dl -f bestaudio+bestvideo "http://.../watch?v=id automatically choose highest quality for download


2016/03/30 VPN,Hosting

I was doing a bit of research for "cloud computing" hosting costs for running Shadowsocks. Amazon Web Services (AWS) is a good option as it's free for the first 12 months when signing up.

For running Shadowsocks the most suitable choice is the small, general cloud computing service "t2.micro" that is free for 12 months (with max 1 instance running). The general cloud computing service is called Amazon Elastic Compute Cloud, normally written Amazon EC2.

Regarding hosting costs I was a bit surprised to see AWS charges different rates for different regions (See I have selected Japan for hosting, but AWS also has servers in Seoul, Singapore, US, Europe etc. The hosting costs is actually quite a lot higher in Asia compared to US/Europe.

On-demand t2.micro linux/unix costs (2016-03-30)

Region Cost (USD) / hour Cost (USD) / month (750 h)
US West (Oregon) 0.013 9.75
US West (Northern California) 0.017 12.75
EU (Frankfurt) 0.015 11.25
Asia Pacific (Tokyo) 0.02 15
Asia Pacific (Seoul) 0.02 15
Asia Pacific (Singapore) 0.02 15

Better prices can be had if buying 24/7 hosting contracts over longer periods. AWS writes: "Reserved Instances provide you with a significant discount (up to 75%) compared to On-Demand Instance pricing."

1-year contract for t2.micro linux/unix at Asia Pacific (Tokyo) (2016-03-30)

Payments Upfront Monthly Effective hourly Effective monthly Savings over On-Demand hourly
No upfront 0 10.95 0.015 10.95 25%
Partial upfront 85 3.65 0.0147 10.73 27%
All upfront 126 0 0.0144 10.5 28%

Its basically not a big difference between paying upfront or not (5.4 USD max savings per year) so for me the best choice is monthly with "no upfront". 3-year contracts are also available, with savings up to 53% (effectively 6.86 USD/month for 3 years upfront).


Technical specs for "t2.micro": 1 vCPU, 1 GB ram, "EBS only" instance storage

From AWS:

T2 instances are Burstable Performance Instances that provide a baseline level of CPU performance with the ability to burst above the baseline. The baseline performance and ability to burst are governed by CPU Credits.

For example, a t2.small instance receives credits continuously at a rate of 12 CPU Credits per hour. This capability provides baseline performance equivalent to 20% of a CPU core. If at any moment the instance does not need the credits it receives, it stores them in its CPU Credit balance for up to 24 hours. If and when your t2.small needs to burst to more than 20% of a core, it draws from its CPU Credit balance to handle this surge seamlessly.

Many applications such as web servers, developer environments and small databases don’t need consistently high levels of CPU, but benefit significantly from having full access to very fast CPUs when they need them. T2 instances are engineered specifically for these use cases. If you need consistently high CPU performance for applications such as video encoding, high volume websites or HPC applications, we recommend you use Fixed Performance Instances.

This is a pretty clever setup for this type of services. The t2.micro gets 6 CPU Credits / hour and runs on Intel Xeon family CPU (up to 3.3 GHz). I guess this means the t2.micro baseline performance is 10% of a Xeon 3.3 GHz CPU.

Binding IP addresses

You can have one Elastic IP (EIP) address associated with a running instance at no charge. If you associate additional EIPs with that instance, you will be charged for each additional EIP associated with that instance per hour on a pro rata basis.

So one IP address is for free which should be enough for Shadowsocks. Here I actually made a mistake, I changed the IP for my instance (as it was working poorly) but I never erased the old one! Turns out one has to pay for Ip addresses not attached to running instances too:

$0.005 per Elastic IP address not attached to a running instance per hour (prorated)

From my monthly bill statement. Turns out you can detach an IP and keep it for free for 1 h, after that AWS will charge you each hour. IP binding remapping are also free for 100 IP bindings/month. 

Data transfer

Data Transfer IN To Amazon EC2 From
Internet $0.00 per GB

Data Transfer OUT From Amazon EC2 To Internet
First 1 GB / month $0.000 per GB
Up to 10 TB / month $0.140 per GB

So here one needs to estimate how much data will actually go via Shadowsocks. If there is a lot of data this will actually be a quite big cost, especially as Youtube is blocked so all that traffic need to go through Shadowsocks.

From my monthly bill I can read that the first 15 GB transfer out was not charged (included in monthly global free tier). The 12 month free tier includes: "15 GB of bandwidth out aggregated across all AWS services".


Amazon Elastic Block Store (Asia Pacific Tokyo) prices:

Amazon EBS General Purpose (SSD) volumes
$0.12 per GB-month of provisioned storage

Amazon EBS Magnetic volumes
$0.080 per GB-month of provisioned storage
$0.080 per 1 million I/O requests 

 As there wont be any data stored for Shadowsocks this isn't an issue.

The 12 month free tier includes: "30 GB of Amazon Elastic Block Storage in any combination of General Purpose (SSD) or Magnetic".


2016/03/29 VPN,OpenWRT

I bought an old Netgear WNDR4300 router to play around with to install customised router firmware. This makes it possible to add a lot of functions a consumer router wouldn't normally have, which can be quite useful, especially here in China where you have the Great Firewαll to cope with.

There are actually made router firmware projects, "DD-WRT", "Tomato" and "OpenWRT" being the more wellknown. OpenWRT is built on Linux (as is DD-WRT and Tomato) but comes with a modular design and package-management system which makes it possible to configure the system according to the user's needs.

Netgear WNDR4300 was actually designed to run a version of OpenWRT as its firmware from the start, so it seems reasonable to assume that since it was designed for OpenWRT, its also a good choice of router to try out newer versions of OpenWRT. It is also one of the recommended routers to run OpenWRT which was the reason I decided to buy this model.

Step 1: Download firmware for WNDR4300

The WNDR4300 has actually been made in different versions, so first step is to check which version it actually is (it was WNDR4300 version 1 in my case). OpenWRT firmware builds are available for many different platforms, depending on CPU and other hardware. The documentation was a bit confusing but after some research I found that the WNDR4300 has a Atheros AR9344 560MHz CPU and belongs to the "ar71xx" OpenWRT platform. 

I downloaded latest stable build, the 15.05.1 (Chaos Calmer) release:
There is also a SquashFS version available, which I think will save RAM on the device as the read-only part of the filesystem can be compressed but I haven't tried that version. The WNDR4300 comes with 128 MB flash + 128 MB RAM memory, which should be quite ok. The OpenWRT website states that the smallest installation requires 4 MB flash + 16 MB ram.

Step 2: Upload firmware to router

There are different ways to do this, but uploading by tftp seems to be recommened. To do this reset the router and put it in listening mode.

  1. Power off the router
  2. Press the reset button and keep holding it down during power up. Keep pressing reset until the power LED starts blinking green.
  3. The router is now running with ip and is waiting for firmware upload
  4. Connect to a LAN port on the back of the router with tp-cable (luckily I have my Macally usb-ethernet converter for my Macbook)
  5. Set a static ip for the computer in 192.168.1.x subnet (i.e. and use as gateway
  6. Upload the firmware to the router. I read somewhere that too long filename may cause problem so I renamed the firmware file “openwrt-15.05.1-ar71xx-nand-wndr4300-ubi-factory.img” to “firmware.img".
    Run the following commands:
    tftp -e
    mode binary
    put firmware.img

Then wait for power LED to turn green again. Go to using web browser and set a new root password. Also set the correct timezone in the System -> System menu.

I also changed the local LAN ip to in order to not interfere with my other router. Need to re-apply the ssh settings under “administration” after updating the LAN ip number (otherwise ssh is refused, guess it expects

Step 3: Install packages

The next step is to install "Shadowsocks" and "ChinaDNS" packages. Many guides online also list Redsocks2 in order to set up Shadowsocks as a transparent proxy but the current version of Shadowsocks has transparent proxy functionality built-in so Redsocks2 is not needed. Note that OpenWRT-shadowsocks comes in two versions - shadowsocks-libev and shadowsocks-libev-spec. I use the later one, it comes with LuCl web interface and does not need Redsocks2.

Shadowsocks SOCKS5 proxy client
ChinaDNS Resolve DNS (Determine if IP is in China or not; avoid DNS pollution etc)

Use terminal and ssh to login to the router:
ssh [email protected]

OpenWRT comes pre-installed with opkg package manager (an OpenWRT fork of ipkg). There is also a "Software" menu in the OpenWRT GUI which can also be used to install packages.

opkg update
opkg install shadowsocks-libev-spec
opkg install ChinaDNS

The commands above may work (opkg download and install packages) but here I run into a problem, because opkg couldn't find any prebuilt packages to install. I tried adding some custom package feeds but since OpenWRT version 15.05 the package manager (opkg) started checking signatures for all packages (which is a good thing), but unfortunately had the repository that keeps Shadowsocks / ChinaDNS no valid signatures!

It should be possible to change the configuration in /etc/opkg.conf from “option check_signature 1” to “option check_signature 0” to avoid the signature check but this didnt work for me for some reason (I later read a post that removing the line will cancel the signature check, but changing to zero doesn't work).

I instead installed them locally, which means I first need to manually download the packages and copy (scp) them over to the router.

The package files are available to download from:

LuCI is the Web User Interface of OpenWRT. Each module has two packages, the actual router module software installation (ar71xx for this platform) and a corresponding GUI plugin.



Copy over all IPK packages to the router /tmp folder using scp:
scp *.ipk [email protected]:/tmp/

Then run installation of all packages. I don't think the installation order should make any difference.
Each package is installed by running "opkg install packagename.ipk".

opkg update
opkg install ChinaDNS_1.3.2-3_ar71xx.ipk
and so on ...

While installing shadowsocks I got an error message: "failed to find a module named nf_tproxy_core". Not sure about what this means, but I read at a forum that a router reboot is enough to solve this problem. I haven't noticed any problems after rebooting (no errors/warnings in kernel or system log files either).

Step 4: Configure software

  1. Input your Shadowsocks server settings (server ip & port, password and encryption). The configuration will be stored in /etc/config/shadowsocks 
  2. Update "DHCP and DNS" settings, see screenshots below. Need to change 2 settings - "DNS forwardings" and "Ignore resolve file" - for ChinaDNS to work correctly.
  3. Turn on "UDP Forward" for Shadowsocks (this may not be available for older versions). I don't run the global "UDP-Relay Server" for Shadowsocks.

Shadowsocks uses port 1080 as default for it's SOCKS5 proxy. Need to fill in server IP & port, password and encryption method. I haven't used "One-time authentication", wasn't sure what this function was, but from Shadowsocks documentation one can read:

One-time authentication (shortened as OTA) is a new experimental feature designed to improve the security against CCA (Chosen-ciphertext attack). 

DNS uses UDP protocol for DNS lookups which may get blocked going outside China (i.e. Google's DNS servers are blocked). To avoid this the UDP packets can be tunneled in a Shadowsocks TCP connection. Shadowsocks will use port 5300 to listen for UDP packets and forward to the public DNS server (run by Google) at port 53. Port 53 is the standard port for DNS.

Bi-directional filter seems to try to solve inconsistencies in IP number for CDN-networks that has servers both in China and abroad. I haven't tried this yet, so I'm not sure when/how this helps.

I use the default settings for ChinaDNS. Regarding the upstreams servers, the server at is DNS server run by China Telecom (located in Nanjing, China) and the server at is Google's DNS server. The CHNRoute file is the IP subsets that ChinaDNS uses to determine if an IP is in China or not.

ChinaDNS is using port 5353 to listen for incoming connections by default. Set "DNS forwardings" to unless you changed this value.

Need to turn on the "Ignore resolve file" setting.

Extra stuff - tester script

Build a script to test connectivity and automatically restart shadowsocks if connection isn't working.
For this to work we first need to install wget (wget is already installed but it is the stripped down busybox version which will not work with the script below)

opkg update
opkg install wget

Create the file /root/tester (make executable chmod 755) and add the script below:

LOGTIME=$(date "+%Y-%m-%d %H:%M:%S")
wget --spider --quiet --tries=1 --timeout=3
if [ "$?" == "0" ]; then
echo '['$LOGTIME'] No Problem.'
exit 0
wget --spider --quiet --tries=1 --timeout=3
if [ "$?" == "0" ]; then
echo '['$LOGTIME'] Problem detected, restarting shadowsocks.'
/etc/init.d/shadowsocks restart
echo '['$LOGTIME'] Network Problem. Do nothing.'


*/10 * * * * /root/tester >> /var/log/shadowsocks_watchdog.log 2>&1
0 1 * * 7 echo "" > /var/log/shadowsocks_watchdog.log

Under scheduled tasks set up a cronjob as the screenshot above. This will run the tester script every 10 minutes to check status and automatically restart Shadowsocks if it detects a problem. A log file will be kept at /var/log/shadowsocks_watchdog.log

Increase DNS lookup speed

Download the files listed in table below from

accelerated-domains.china.conf Faster lookup of China domains
bogus-nxdomain.china.conf Certain China ISP return unwanted redirects when domain is not found
google.china.conf Speed up access to Google servers in China

Create the folder /etc/dnsmasq.d and copy all files to that folder 
Edit dnsmasq configuration file /etc/dnsmasq.conf and add the line below: