May 21, 2014
TechnologyInstall Software
Internet
Chromium, we need this browser absolutesly:
$ sudo pacman -S chromium
Select 1/8
Oh, we forget install X, so first we will install X:
X Window
Install xorg first:
$ sudo pacman -S xorg xorg-xinit
Awesome Window Manager;
$ sudo pacman -S awesome
Edit the .xinitrc file, add following lines:
exec awesome
Necessary video driver:
$ sudo pacman -S xf86-video-intel xf86-video-ati
Continue Internet
Firefox, another browser, pidgin, for chatting, thunderbird for email, wget for downloading:
sudo pacman -S firefox pidgin thunderbird wget
Office
Libreoffice,need download 253MB, so take a break:
$ sudo pacman -S libreoffice
# choose Enter-> 25-> Enter
Terminals
Install xterm first, initial term:
$ sudo pacman -S xterm
xfce4-terminal will be installed in next chapterVncserver.
gnome-terminal is a good option:
$ sudo pacman -S gnome-terminal
Vncserver
Install tigervnc
$ sudo pacman -S tigervnc
Use xfce4 for remote vnc(For many users may not familiar with awesome desktop):
$ sudo pacman -S xfce4
# choose (default=all)
Configure vncserver:
$ cat ~/.vnc/xstartup
#!/bin/sh
export XKL_XMODMAP_DISABLE=1
exec startxfce4
With VNC, you can start up the X and let the chromium sync up its bookmarks.
Documents
Install evince for viewing pdf and other documents:
$ sudo pacman -S evince
Install gimp for processing pictures:
$ sudo pacman -S gimp
Video/Fun
Install smplayer:
sudo pacman -S smplayer
Without ALSA, you can do nothing, so install it.
sudo pacman -S alsa-utils
Development
Install gvim and eclipse:
$ sudo pacman -S gvim eclipse
Git and Subversion:
$ sudo pacman -S git subversion
Wireshark and Tcpdump:
$ sudo pacman -S wireshark-gtk tcpdump
ddd and gdb for debugging:
$ sudo pacman -S ddd gdb
meld for comparing files:
$ sudo pacman -S meld
Virtualization
Install qemu and virtualbox:
sudo pacman -S virtualbox qemu
Chinese Localization
Install fonts for chinese:
$ sudo pacman -S wqy-bitmapfont wqy-microhei wqy-microhei-lite wqy-zenhei
Install fcitx input method:
$ sudo pacman -S fcitx fcitx-libpinyin fcitx-configtool fcitx-qt4 fcitx-qt5
Configuration will be wrote later.
rox for file browser:
$ sudo pacman -S rox
gpicview for picture viewer:
$ sudo pacman -S gpicview
Conky for viewing the tasks:
$ sudo pacman -S conky
.conky file will be downloaded from the github(Later I will upload).
.xinitrc file will be uploaded to github.
Now step2 is OK, we will use usd-disk for rebooting.
May 19, 2014
Technology准备
RaspberryPI, SD卡（4G以上), 移动硬盘，操作系统镜像文件，最好有一个外接供电带电路隔离的USB HUB。 鼠标、键盘等。
用SD卡启动
将SD卡插入电脑，将镜像文件写入到SD卡后，将写好的SD卡插入RaspberryPI，加电开机。各种配置（譬如显存大小，是否启动到X等等）完成之后，进入到Linux桌面。
准备硬盘
将硬盘插入USB口，如果之前有分好区的，可以略过这一节，直接到拷贝至硬盘一节。
在RaspberryPI系统里(wheezy or archLinux)，搜索gparted, 这个工具可以在图形化界面下对硬盘进行分区。
命令行下你可以通过下列命令查看已挂载的存储设备信息：
[root@alarmpi ~]# fdisk -l
Disk /dev/mmcblk0: 7.3 GiB, 7822376960 bytes, 15278080 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00047c7a
Device Boot Start End Blocks Id System
/dev/mmcblk0p1 8192 122879 57344 c W95 FAT32 (LBA)
/dev/mmcblk0p2 122880 15278079 7577600 83 Linux
Disk /dev/sda: 465.8 GiB, 500105740288 bytes, 976769024 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x0004f23a
Device Boot Start End Blocks Id System
/dev/sda1 2048 1050623 524288 82 Linux swap / Solaris
/dev/sda2 1050624 126879743 62914560 83 Linux
/dev/sda3 126879744 546310143 209715200 7 HPFS/NTFS/exFAT
/dev/sda4 546310144 976769023 215229440 5 Extended
/dev/sda5 546312192 588255231 20971520 83 Linux
/dev/sda6 588257280 976769023 194255872 83 Linux
上面的代码中可以看到，我的RaspberryPI上挂载的硬盘大小是500G。分成了若干个区。
拷贝文件系统到硬盘分区
分区决定于你自己，用gparted或者别的分区工具都可以分好，记得选择Linux ext4为文件系统。
这里我选择/dev/sda5作为我的根分区，所以在终端上，执行下列命令：
$ mkfs.ext4 /dev/sda5
$ sudo mount /dev/sda5 /mnt
$ cd /mnt
$ tar cjvf /mnt/rootfs.tar.bz2 /
备份完毕后，解压：
$ tar xjvf /mnt/rootfs.tar.bz2 -C /mnt/
这里用tar备份文件系统时主要是为了防止系统中某些可能的链接/设备文件的丢失，干脆先压缩后再解压，一劳永逸，代价是时间会花多一点。用rsync是既保险又快速的方法，如果你系统上安装了rsync，试下下面的命令：
rsync -rv --exclude=/mnt / /mnt
总之，这一步的目的是把SD卡上的文件系统完整复制到硬盘分区上。
修改启动选项
修改SD的boot分区上内容，文件cmdline.txt以使得其适配硬盘启动：
$ cd /boot
$ cp cmdline.txt cmdline.txt.SD
$ >cmdline.txt
$ echo "smsc95xx.turbo_mode=N dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=/dev/sda5 rootfstype=ext4 elevator=noop rootwait">cmdline.txt
上面命令的作用是把root=参数从root=/dev/mmcblk0p2换到root=/dev/sda5，你的硬盘分区可能不是sda5，这取决于你在上两步中采用了哪块分区。例如sda(1?2?3?4?)，这里的数值和你在上两步中应该保持一致。
如果你的移动硬盘上创建了交换分区，即swap分区，记得在/etc/fstab文件中加入相应的条目，下面是的系统上，用sda1作为swap分区时的配置
# cat /etc/fstab
#
# /etc/fstab: static file system information
#
# <file system> <dir> <type> <options> <dump> <pass>
/dev/mmcblk0p1 /boot vfat defaults 0 0
/dev/sda1 none swap defaults 0 0
为了确保写入成功，在操作的最后，麻烦运行一下:
$ sudo sync
这可以保证所有的外接设备上的数据被安全写入。
没有问题的话，现在重新启动RaspberryPI，你的文件系统就应该跑在硬盘上了。
如果有失败的话，把SD卡插入电脑，把cmdline.txt文件恢复成cmdline.txt.SD文件即可。
特别强调
特别强调的一点，彻底从硬盘启动是不可能的！！！也就是说，你依然需要插入一张SD卡在RaspberryPI的槽中。因为RaspberryPI每次都是从固定地址读取启动代码的。
上述做法的好处是避免了频繁读写SD卡，延长了SD卡的使用寿命。
缺点是： USB硬盘的速度，最多也就是USB2.0的极速。 SD卡，CLASS10的速度可能要超过USB2.0的速度。某些时候会慢一点，但是秒杀CLASS4的卡是没问题的。
May 14, 2014
TechnologyInstallation
Via following command
$ yaourt -S nodejs
Quick Start
$ node
> console.log("Hello!")
Hello!
undefined
Hit twice “Ctrl+c” you will get out of the terminal.
A simple example is like:
var http = require("http");
http.createServer(function(request, response) {
response.writeHead(200, {"Content-Type": "text/plain"});
response.write("Hello World");
response.end();
}).listen(8888);
$ node server.js
$ curl http://localhost:8888
Hello World%
May 13, 2014
TechnologyUse Template In Flask
To use template in flask, we should put the static file under the templates folder under the root directory. Our index page should looks like following:
So our html file shall wrote like following:
<!DOCTYPE HTML>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>{{title}}</title>
<link rel="stylesheet" type="text/css" href="/static/assets/css/styles.css" />
<!--[if IE]><script src="assets/js/excanvas.min.js"></script><![endif]-->
</head>
<body>
<div id="page">
<div id="header">
<h1>{{title}}</h1>
<h2>Nanjing Weather/PM Statistics</h2>
<div id="periodDropDown">
<span class="left"></span>
<span class="currentPeriod">Last 24 hours</span>
<span class="arrow"></span>
<span class="right"></span>
<ul>
<li data-action="24hours">Last 24 hours</li>
<li data-action="7days">Last 7 Days</li>
<li data-action="30days">Last 30 Days</li>
</ul>
</div>
</div>
<div class="temperature section">
<h3>Temperature</h3>
<div id="plot">
<span class="preloader"></span>
</div>
</div>
<div class="humidity section">
<h3>Humidity</h3>
<div id="humi_plot">
<span class="preloader"></span>
</div>
</div>
<div class="pm2.5 section">
<h3>PM2.5</h3>
<div id="pm25_plot">
<span class="preloader"></span>
</div>
</div>
<div class="pm10 section">
<h3>PM10</h3>
<div id="pm10_plot">
<span class="preloader"></span>
</div>
</div>
</div>
<p id="footer">
{{year}} © {{title}}. Powered by <a href="kkkttt.github.io">Dash</a> UTC: {{utctime}}
</p>
<script src="/static/assets/js/jquery.min.js"></script>
<script src="/static/assets/js/script.js"></script>
<script src="/static/assets/js/jquery.flot.min.js"></script>
</body>
</html>
To use CSS file to make sure the vision effect, to use css in flask, put your files into the directory static under the root directory. Just like following :
$ tree static
static
└── assets
├── css
│ └── styles.css
├── img
│ ├── bg_tile.jpg
│ ├── bg_vert.jpg
│ ├── preloader.gif
│ └── sprite.png
└── js
├── jquery.flot.min.js
├── jquery.min.js
└── script.js
Rendering Html
In genhtml.py, we define the function which rendering the html template like following:
# @app.route('/index')
@app.route('/')
# Generate the index page, for debugging now
def index():
# Use template for rendering the content
Current_Year = datetime.now().strftime("%Y::%H:%M ")
Current_UTC = datetime.utcnow().strftime("%H:%M")
return render_template('index.html', title="NanJing Weather and PM2.5/10 Statistics", year=Current_Year, utctime=Current_UTC)
Now open your first page, you will see the rendered effect. But the flot div remains empty, next chapter we will introduce the javascript which used for draw the flot picture and AJAX which used for updating the content.
May 13, 2014
TechnologyDraw Flot Picture
Since the article is mainly on writing apps, I don’t want to spend much time on how to use javascript or flot for drawing picture.
Simply checkout the code on github, you will see the code which is used for retrieving the data and start drawing plot pictures.
Fetching 24-hours Data
Fetching 24 latest records from the postgres database, and then add them into the chart, chart then has been sent to simplejson which used for updating the flot picture locally .
@app.route('/ajax/24hours/')
# ajax updateing for 24-hour data
def TwentyFourHours():
# Here we will visit postgres for retrieving back the last 24 hours' data
# Local version
# engine = create_engine('postgresql+psycopg2://Trusty:@localhost:5432/mylocaldb', echo=True)
# Heroku Version
db_conn = os.environ['DATABASE_URL']
engine = create_engine(db_conn)
# Engine selection
metadata=MetaData(bind=engine)
weather_table=Table('weather',metadata,
Column('Insert_Time', DateTime(timezone=True),primary_key=True),
Column('Temperature', Integer),
Column('Humidity', Integer),
Column('PmTen', Integer),
Column('PmTwoFive', Integer),
)
s = select([weather_table]).order_by(weather_table.c.Insert_Time.desc()).limit(24)
conn = engine.connect()
result = conn.execute(s)
chart = []
for row in result:
# row[0], datetime;
# row[1], Temperature;
# row[2], Humidity;
# row[3], PM10;
# row[4], PM2.5;
### append row[x] into the char ###
# chart.append({
chart.insert(0, {
"label": (row[0] + timedelta(hours = 8)).strftime("%H:%M"),
"value": row[1],
"humi_value": row[2],
"pm25_value": row[4],
"pm10_value": row[3]
})
jsonStr = simplejson.dumps({
# This is char, will be used in script.js
"chart" :{
# tooltip is used by the jQuery chart:
"tooltip" : "Temperature at %1: %2 degree",
# humi_tooltip is used for jQuery chart for humidity:
"humi_tooltip" : "Humidity at %1: %2 \%",
"pm25_tooltip" : "PM2.5 at %1: %2 ug/m(3)",
"pm10_tooltip" : "PM10 at %1: %2 ug/m(3)",
"data" : chart
},
# This is "downtime" will be used in script.js
"downtime" : getDowntime(1)
})
return jsonStr;
The periodic task which runs in tasks.py will fetching the data from the python api and the webpage, then insert them into the postgres database, so here in 24-hours’ function we just get them out and fill in the flot picture.
Daily Data
Daily data shall be generated via calculating them at the mid-night, that is, at the beginning of a brand new day, we will calculate out the last day’s average data.
The code is implemented in tasks.py as a crontab task, the code is listed as following:
# Every Day we will run a periodly work which will calculate
# the average temperature/humidity/pm2.5/pm10 task, and store
# it into the daily database, thus we have to define new Database
# here and insert data into.
# Beijing is locate at east 8 zone, thus 16:12 + 8 hour = 24:12
# At every mid-night(24:12) it will caculate the average value for
# the past 24 hours.
@periodic_task(run_every=crontab(hour=16, minute=12))
def OneDayHandler():
# First Get the last 24 hours' data set
# Local Engine
# engine = create_engine('postgresql+psycopg2://Trusty:@localhost:5432/mylocaldb',echo=True)
# Heroku Engine
db_conn = environ['DATABASE_URL']
engine = create_engine(db_conn)
metadata=MetaData(bind=engine)
# Definition of the weather table
weather_table=Table('weather',metadata,
Column('Insert_Time', DateTime(timezone=True),primary_key=True),
Column('Temperature', Integer),
Column('Humidity', Integer),
Column('PmTen', Integer),
Column('PmTwoFive', Integer),
)
# Get last 24 records, If we suppose there are truely 24 records in last 24 hours, we can enable this sentense. But sometimes, this will be wrong.
s = select([weather_table]).order_by(weather_table.c.Insert_Time.asc()).limit(24)
conn = engine.connect()
results = conn.execute(s)
# Temperature
totTemperature = 0
avgTemperature = 0
# Humidity
totHumidity = 0
avgHumidity = 0
# PM2.5
totPm25 = 0
avgPm25 = 0
# PM10
totPm10 = 0
avgPm10 = 0
records_number = 0
for item in results:
totTemperature += item[1]
totHumidity += item[2]
totPm25 += item[4]
totPm10 += item[3]
records_number += 1
if records_number>0:
avgTemperature = totTemperature/records_number
avgHumidity = totHumidity/records_number
avgPm25 = totPm25/records_number
avgPm10 = totPm10/records_number
# Definition of the avg_eather table
avg_metadata = MetaData(bind=engine)
avg_weather_table=Table('avg_weather',avg_metadata,
Column('avg_Insert_Time', DateTime(timezone=True),primary_key=True),
Column('avg_Temperature', Integer),
Column('avg_Humidity', Integer),
Column('avg_PmTen', Integer),
Column('avg_PmTwoFive', Integer),
)
# Create table in db
avg_metadata.create_all(checkfirst=True)
# Create insert sentense
avg_ins = avg_weather_table.insert()
# Really insert
avg_ins = avg_weather_table.insert().values(avg_Insert_Time=datetime.utcnow(), avg_Temperature = avgTemperature, avg_Humidity = avgHumidity, avg_PmTen = avgPm10, avg_PmTwoFive = av
gPm25)
# Connect to engine and execute
avg_conn = engine.connect()
avg_conn.execute(avg_ins)
return 1
We defined a new table named avg_weather, and at the mid-night we will retrieve the latest 24 records, calculating their average value, then insert them into the aver_weather table.
Displaying Daily Data
The main procedure is mainly like in 24-hours datas, but notice we are select from avg_weather, and we only select 7 items.
30-days data is very simple, change the day from 7 to 30, then you can see the monthly data.
Write Testing Interface
We hope we can manually test the functions via web. So we added following testing APIs in genhtml.py:
@app.route('/test/fetch/')
def fetch():
fetch_and_store_data()
return "Fetching Test Done!!!";
@app.route('/test/gen/')
def generateOneDay():
OneDayHandler()
return "Generate Test Done!!!";
If we visit http://Your_app_address/test/fetch, the program will fetch back the data. And for http://Your_app_address/test/gen, the daily average data will be generated.
Next Chapter is the last one. We simply paste the screenshots of the APP.
