Alfred's spAcE

2016.05.17

ss-redir

Filed under: Linux, misc — Tags: , , , , — Alfred Yang @ 20:20

I’ve just setup ss-redir on my Linksys E3000.

  1. Install entware on my router, it is using shibby tomato rom;
  2. Install shadowsocks-libev;
  3. Refer to here to setup iptables rules; I don’t redirect UDP traffic;
  4. If your DNS will be poisoned, you should manage to fix it first;

Let’s f**k GFW.

Some notes:

I prefer to use ipset, so we don’t need many iptable rules, below is my part of iptables setup script:

#!/bin/sh

# clear ss rules
iptables-save -c | grep -v "SHADOWSOCKS" | \
 grep -v "gfwlist" | \
 iptables-restore -c 2>/dev/null

IPTA='iptables -t nat -A SHADOWSOCKS'

# create ss chain in table - NAT
iptables -t nat -N SHADOWSOCKS

# ignore not in gfwlist
$IPTA -m set ! --set gfwlist dst -j RETURN

# redirect ss-redir
$IPTA -p tcp -j REDIRECT --to-ports 1081

# redirect NAT output to ss chain
iptables -t nat -I PREROUTING -p tcp -j SHADOWSOCKS

# redirect local output traffic to ss chain
iptables -t nat -I OUTPUT -p tcp -j SHADOWSOCKS

Of coz, using ipset imply that your router has ipset kernel module; I add below init script to my router:

for module in ipt_REDIRECT ip_set ipt_set ip_set_nethash ip_set_iphash 
do
 modprobe $module
done
ipset -N gfwlist iphash

My dnsmasq can support gfwlist, so ipset “gfwlist”‘s entries do not need to manual enter, but my dnsmasq config file will be a bit big. Below is part of my script to generate DNS and ipset related things:

#!/bin/sh
BASE=/opt/shadowsocks
DNSMASQ=$BASE/dnsmasq.d
GFWLIST=$DNSMASQ/gfwlist.conf
TMPFILE=/tmp/gfwlist.tmp 
TMPLIST=/tmp/gfwlist.conf
URL=<some url to get gfwlist.conf>
# below change to your secure DNS
DNS=127.0.0.1#5353

set -e

[ -d $DNSMASQ ] || mkdir -p $DNSMASQ

curl -s $URL > $TMPFILE

# change to our dns
sed -i "s|^\(server.*\)/[^/]*$|\1/$DNS|" $TMPFILE

echo . > $TMPLIST
grep "^server" $TMPFILE >> $TMPLIST
grep "^ipset" $TMPFILE >> $TMPLIST
sed "1d" -i $TMPLIST

# update conf file
mv $TMPLIST $GFWLIST
echo Update done.

# flush ipset
echo flush ipset.
ipset -F gfwlist

# restart dnsmasq
echo restart dnsmasq.
service dnsmasq restart

 

2016.02.10

decorator in python

Filed under: misc — Tags: — Alfred Yang @ 20:55

I write a script to check-in a web service; it consists of 1~3 http gets, and maybe one http post to login my account. It is very handy, I can use crontab to let it run by schedule, so if my computer is running, I won’t forget to check-in.

I often see the requests lib complains SSLError – “Unexpected EOF”; I’m not sure what’s the problem, but usually a retry can solve this problem. So I think of add a retry mechanism to all these gets and posts. The retry codes will be like this:

_retry_cnt = 3
while _retry_cnt > 0:
    try:
        # gets/posts
        break
    except:
        _retry_cnt -= 1
else:
    # too many error, quit
    quit()

But as you can see, I have to copy them to replace all gets/posts, it does not look like an good idea. After look up in python language reference, find decorator; so I don’t need to change all gets/posts, I just need to add a decorator before my gets/posts functions. I will be like this:

# decorator function to do 3 times retries
def web_try(Func):
    def action(*args, **kwds):
        _retry_cnt = 3 
        while _retry_cnt > 0:
            try:
                Func(*args, **kwds)
                break
            except:
                myprint('exception count down')
               _retry_cnt -= 1
        else:
            myprint('too many retries, abort mission!')
            quit()
    return action

# two functions need decorator
@web_try
def web_get(url):
    b.open(url) # b is a robobrowser object

@web_try
def web_auth(url, payload):
    b.session.post(url, data=payload)

So I don’t need to need to do code changes in main logic.

2015.12.21

Firefox in LXC

Filed under: Linux, misc — Tags: , — Alfred Yang @ 19:49

This is referring to https://www.stgraber.org/2014/02/09/lxc-1-0-gui-in-containers/

## I use Trusty in Wily
$ lxc-create -t download -n gui -- -d ubuntu -r trusty -a amd64

## change mount and pre-start hook(replacing USERNAME appropriately):
$ vi ~/.local/share/lxc/gui/config
lxc.mount.entry = /dev/dri dev/dri none bind,optional,create=dir
lxc.mount.entry = /dev/snd dev/snd none bind,optional,create=dir
lxc.mount.entry = /tmp/.X11-unix tmp/.X11-unix none bind,optional,create=dir
lxc.mount.entry = /dev/video0 dev/video0 none bind,optional,create=file

lxc.hook.pre-start = /home/USERNAME/.local/share/lxc/gui/setup-lxc.sh

# change id_map, same file
lxc.id_map = u 0 100000 65536
lxc.id_map = g 0 100000 65536
-> (assuming uid/gid is 1000/1000)
lxc.id_map = u 0 100000 1000
lxc.id_map = g 0 100000 1000
lxc.id_map = u 1000 1000 1
lxc.id_map = g 1000 1000 1
lxc.id_map = u 1001 101001 64535
lxc.id_map = g 1001 101001 64535

## create start-up hook
$ vi ~/.local/share/lxc/gui/setup-lxc.sh
#!/bin/sh
PULSE_PATH=$LXC_ROOTFS_PATH/home/ubuntu/.pulse_socket

if [ ! -e "$PULSE_PATH" ] || [ -z "$(lsof -tn $PULSE_PATH 2>&1)" ]; then
    pactl load-module module-native-protocol-unix auth-anonymous=1 \
        socket=$PULSE_PATH
fi

# below to let container can connect to host X server
XAUTH_FILE="${LXC_ROOTFS_PATH}/home/ubuntu/.Xauthority"
rm $XAUTH_FILE
touch $XAUTH_FILE
xauth extract - $DISPLAY | xauth -f $XAUTH_FILE merge ~/.Xauthority

## after create script, make it executable
chmod a+x ~/.local/share/lxc/gui/setup-lxc.sh
## change folder owner
$ sudo chown -R 1000:1000 ~/.local/share/lxc/gui/rootfs/home/ubuntu

## install software in container
$ lxc-start -n gui -d
$ lxc-attach -n gui -- umount /tmp/.X11-unix
# you can change ubuntu repo to your prefer site;
$ lxc-attach -n gui -- apt-get update
$ lxc-attach -n gui -- apt-get dist-upgrade -y
$ lxc-attach -n gui -- apt-get install ubuntu-artwork dmz-cursor-theme ca-certificates pulseaudio firefox -y
$ lxc-attach -n gui -- apt-get -f install -y
$ lxc-attach -n gui -- sudo -u ubuntu mkdir -p /home/ubuntu/.pulse/
$ echo "disable-shm=yes" | lxc-attach -n gui -- sudo -u ubuntu tee /home/ubuntu/.pulse/client.conf
# optional install - tmpreaper fonts-wqy-microhei flashplugin-installer default-jre icedtea-7-plugin
$ lxc-stop -n gui

## script to start firefox in LXC
$ vi ~/.local/share/lxc/gui/start-firefox 
#!/bin/sh
CONTAINER=gui
CMD_LINE="firefox $*"

STARTED=false

if ! lxc-wait -n $CONTAINER -s RUNNING -t 0; then
 lxc-start -n $CONTAINER -d
 lxc-wait -n $CONTAINER -s RUNNING
 STARTED=true
fi

PULSE_SOCKET=/home/ubuntu/.pulse_socket

lxc-attach --clear-env -n $CONTAINER -- sudo -u ubuntu -i \
 env DISPLAY=$DISPLAY PULSE_SERVER=$PULSE_SOCKET $CMD_LINE

if [ "$STARTED" = "true" ]; then
 lxc-stop -n $CONTAINER -t 10
fi

## desktop file (replacing USERNAME appropriately):
$ vi ~/.local/share/applications/lxc-firefox.desktop 
[Desktop Entry]
Version=1.0
Name=Firefox in LXC
Comment=Access the Internet
Exec=/home/USERNAME/.local/share/lxc/gui/start-firefox %U
Icon=/home/USERNAME/.local/share/lxc/gui/rootfs/usr/share/pixmaps/firefox.png
Type=Application
Categories=Network;WebBrowser;

Import non-ubuntu image in lxd

Filed under: Linux, misc — Tags: — Alfred Yang @ 13:08

Now, lxd-images can only import Ubuntu images from Ubuntu cloud images.

$ lxd-images import -h
usage: lxd-images import [-h] {busybox,ubuntu} ...

positional arguments:
 {busybox,ubuntu}
 busybox Busybox image
 ubuntu Ubuntu images

optional arguments:
 -h, --help show this help message and exit

However, we still can see a lot of lxd images on https://images.linuxcontainers.org, how to copy CentOS image from it?

We can add remote site by “lxc remote add”, then you can use “lxc image copy” to copy remote image to your local.

2015.11.18

Mac OS X in VBox in Ubuntu in Thinkpad

Filed under: Linux, misc — Tags: , , — Alfred Yang @ 20:22

mac-in-vb

2015.10.24

Upgrade to Ubuntu Wily

Filed under: Linux — Tags: — Alfred Yang @ 11:07

Ubuntu Wily is released. I have to admit that I should not upgrade so early…

The whole upgrade process is “smooth”:

  1. disable zram_tmp & zram_swap service, in case they cause something;
  2. remove 3rd party repo list; yes, the upgrade process will do it for you, but I don’t like the comments it adds;
  3. remove my /etc/apt/apt.conf.d/01ubuntu-prefer, because I also add Wily repo when I’m using Vivid, and I want to prefer Vivid packages of coz; firstly I forget, so the upgrade always crash;

So, let’s see problems:

  1. Oops, my kernel is still 3.19.0.26; see this bug;
  2. There is no default route after I connect my ppp connection with “pon myconnection”, even the config is ok under Vivid; I have to add a script in /etc/ppp/ip-up.d/ to add it.
  3. Ahhh, finally this my pain – Thinkpad T440p’s clickpad, I hate it. I’m using this patched driver to gain middle button scroll. So now there is no ppa package for Wily yet. I have to download Vivid original source, this patched driver source, and Wily’s package source, do a 3-way merge, build it by myself – take a note, use “dpkg-buildpackage -rfakeroot -uc -b” to build .deb files.

2015.10.17

Upgrade ruTorrent

Filed under: Computers and Internet, Linux — Tags: — Alfred Yang @ 10:26

It is always a headache for me to upgrade it after so many months.

I have to take a note here.

  1. cannot use symbolic link for rutorrent top folder;
  2. (optional)remove unused plugins before move to new versions; I only use _getdir, rss, edit;
  3. chown -R www-data:www-data <rutorrent folder>
  4. stop rtorrent first;
  5. mv/cp new version to correct destination;
  6. start rtorrent; login webpage; make new settings available; stop rtorrent again;
  7. copy old RSS plugin settings to new version: cp -r <old folder>/share/users/admin/settings/rss/cache <new folder>/share/users/admin/settings/rss
  8. start rtorrent and enjoy.

Upgrade openssl and etc. on MyBook Live

Filed under: Computers and Internet, Linux — Tags: — Alfred Yang @ 10:17

MBL is running on a very old Debian Squeeze.

Long time ago, I found the rtorrent my compiling on this box wit rutorrent cannot get some https sites’ RSS. If I use wget/curl on the box to fetch same https URL, it complains something related to TLS. After googling, it turns out that squeeze official package of libcurl and openssl is too old, which does not support TLS 1.1 & 1.2.

So I compile openssl 1.0.1p and curl 7.38 and install, of coz, I have to recompile libtorrent and rtorrent to install again.

It works. The last problem is https tracker…seems the only solution is add “network.http.ssl_verify_peer.set=0” to “.rtorrentrc” (also powered by Google).

2015.07.04

Change the alias of lxc image

Filed under: Linux — Tags: , , — Alfred Yang @ 22:00

After using lxd-images to import some images from linuxcontainers.org, I think the alias I originally use is too long. But it takes me some time to find a way to change the alias. Finally, I find I can add an alias, then delete the old one.
$ lxc image list
+---------------+--------------+--------+-------------+--------+------------------------------+
| ALIAS | FINGERPRINT | PUBLIC | DESCRIPTION | ARCH | UPLOAD DATE |
+---------------+--------------+--------+-------------+--------+------------------------------+
| ubuntu-trusty | 04aac4257341 | no | | x86_64 | Jul 4, 2015 at 8:29am (CST) |
| centos6 | afae698680fc | no | | x86_64 | Jul 4, 2015 at 10:28am (CST) |
+---------------+--------------+--------+-------------+--------+------------------------------+
$ lxc image alias create trusty 04aac4257341
$ lxc image list
+-----------------+--------------+--------+-------------+--------+------------------------------+
| ALIAS | FINGERPRINT | PUBLIC | DESCRIPTION | ARCH | UPLOAD DATE |
+-----------------+--------------+--------+-------------+--------+------------------------------+
| trusty (1 more) | 04aac4257341 | no | | x86_64 | Jul 4, 2015 at 8:29am (CST) |
| centos6 | afae698680fc | no | | x86_64 | Jul 4, 2015 at 10:28am (CST) |
+-----------------+--------------+--------+-------------+--------+------------------------------+
$ lxc image alias delete ubuntu-trusty
$ lxc image list
+---------+--------------+--------+-------------+--------+------------------------------+
| ALIAS | FINGERPRINT | PUBLIC | DESCRIPTION | ARCH | UPLOAD DATE |
+---------+--------------+--------+-------------+--------+------------------------------+
| trusty | 04aac4257341 | no | | x86_64 | Jul 4, 2015 at 8:29am (CST) |
| centos6 | afae698680fc | no | | x86_64 | Jul 4, 2015 at 10:28am (CST) |
+---------+--------------+--------+-------------+--------+------------------------------+

2011.10.27

win7下无法正常删除exe文件

Filed under: Computers and Internet — Tags: — Alfred Yang @ 21:51

    不知从什么时候开始,我的win7 x64 ultimate就老是这个毛病,删除一个刚刚运行(不知道和运行与否有关)的exe程序,点删除(shift-delete)ok,文件消失了,然而F5刷新一下,这个文件又回来了。反反复复删不掉,放着它不管,那么过几分钟,可能会正常消失掉。这个毛病曾经有一阵子导致我的chrome无法正常升级。我也上网google了好久,中文的网页基本没有结果。今天总算是找到了问题所在,在google英文网页的结果,然后还要多看几个,才找到原因——由于Application Experience这个服务被我禁用导致。把AE服务设置成Manual也行,它会自动启动的。

It’s a weird problem – win7 x64 ultimate. When I permanently delete a exe file(such as shift-delete), the file will disappear, but if F5 to refresh the folder, the file reappear. Unless I leave the file a few minutes later. It will be deleted finally. This problem ever prevented me from upgrading my Chrome browser. Now from Google, I find the problem – service “Application Experience” is disabled. I leave this service as manual start. Problem goes away.

http://superuser.com/questions/234569/windows-7-delayed-file-delete

Older Posts »