git(/ɡɪt/,关于这个音频文件 音频(帮助 · 信息))是一个分布式版本控制软件,最初由林纳斯 · 托瓦兹(Linus Torvalds)创作,于 2005 年以 GPL 发布。最初目的是为更好地管理 Linux 内核开发而设计。应注意的是,这与 GNU Interactive Tools[6](一个类似 Norton Commander 界面的文件管理器)有所不同。
1. 还没 add
2.add 但是还没 commit
3. 已经 commit
在本地删除文件,有两种选择:
利用 github 作为免费托管的 git 仓库~
PS: 每次使用命令提交进行版本控制很麻烦,可以选择工具:
附上 mac 简单命令:
学习地址:
实战:
使用 urllib2 库和正则表达式抓取静态新闻网站十篇英文新闻,并统计每篇文章 tf-idf 的 top10 保存到 txt 文件中:
刚学了几天 python 写的,代码写得很烂:
- #coding = utf - 8 import math import json#coding = utf - 8 import urllib import urllib2 import re import string import sys import os url = ['http://www.chinadaily.com.cn/china/2016-04/12/content_24471420.htm'] url.append('http://www.chinadaily.com.cn/china/2016-04/12/content_24470515.htm') url.append('http://www.chinadaily.com.cn/business/2016-04/12/content_24457629.htm') url.append('http://www.chinadaily.com.cn/business/2016-04/09/content_24397697.htm') url.append('http://www.chinadaily.com.cn/business/2016-04/09/content_24397290.htm') url.append('http://www.chinadaily.com.cn/business/chinadata/2016-03/16/content_23898438.htm') url.append('http://www.chinadaily.com.cn/business/2016-04/12/content_24460897.htm') url.append('http://www.chinadaily.com.cn/business/2016-04/12/content_24470803.htm') url.append('http://www.chinadaily.com.cn/business/2016-04/12/content_24455437.htm') url.append('http://www.chinadaily.com.cn/business/2016-04/12/content_24481250.htm') user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'headers = {
- 'User-Agent': user_agent
- }
- try: for i in range(10) : request = urllib2.Request(url[i], headers = headers) response = urllib2.urlopen(request) content = response.read().decode('utf-8') title_pattern = re.compile('div.*?Title_e">(.*?)</div>', re.S) title_items = re.findall(title_pattern, content) title_mypat = re.compile(r '<h1.*?>(.*?)</h1>') title_myitem = re.findall(title_mypat, title_items[0]) filename = str(i + 1) + '.txt'print filename f = open(filename, 'w+') f.write(title_myitem[0]) f.write('\n') pattern = re.compile('div.*?Content">.*?<table.*?</table>(.*?)<div.*?width', re.S) items = re.findall(pattern, content) mypat = re.compile(r '.*?<p>(.*?)</p>.*?') if (items.__len__() > 0) : myitem = re.findall(mypat, items[0]) len = myitem.__len__()#a = []#
- for j in range(len) : #a.append(re.split(r ' |,|\(|\)|\.', str(myitem[j])))#print a
- for j in range(len) : f.write(myitem[j]) except urllib2.URLError,
- e: if hasattr(e, "code") : print e.code
- if hasattr(e, "reason") : print e.reason t = 0 Dict_idf = {}
- Dict_tf = {}
- DicList = []
- for j in range(10) : f_name = str(j + 1) + '.txt'f = open(f_name) mystr = f.read() f.close()#print mystr.__len__() len = mystr.__len__()#print len str_p = ""myarticle = []
- for i in range(len) : if mystr[i] == '.'or mystr[i] == ',': if i1 and mystr[i + 1] >= '0'and mystr[i + 1] <= '9': str_p = str_p.__add__(mystr[i]) elif str_p == "": continue
- else: #print str_p myarticle.append(str_p) str_p = ""elif mystr[i] == '('or mystr[i] == ')'or mystr[i] == ' 'or mystr[i] == '"': if str_p == "": continue
- else: #print str_p myarticle.append(str_p) str_p = ""
- else: str_p = str_p.__add__(mystr[i])#print myarticle.__len__() Dict_tf = {}
- for i in range(myarticle.__len__()) : if not Dict_tf.has_key(myarticle[i]) : Dict_tf[myarticle[i]] = 1.0
- if not Dict_idf.has_key(myarticle[i]) : Dict_idf[myarticle[i]] = 1.0
- else: Dict_idf[myarticle[i]] += 1.0
- else: Dict_tf[myarticle[i]] += 1.0#print myarticle[i],
- Dict_tf[myarticle[i]]#print Dict_tf.__len__() DicList.append(Dict_tf) for k,
- v in Dict_tf.items() : Dict_tf[k] = Dict_tf[k] / myarticle.__len__()#
- for k,
- v in Dict_tf.items() : #print k,
- v
- for k,
- v in Dict_idf.items() : Dict_idf[k] = math.log(10.0 / Dict_idf[k])#
- for k,
- v in Dict_idf.items() : #print k,
- v res = ""res = res.__add__("{") res = res.__add__("\n") res = res.__add__("\t[\n") for j in range(10) : Dictf_idf = DicList[j]
- for k,
- v in Dictf_idf.items() : Dictf_idf[k] = Dictf_idf[k] * Dict_idf[k] Dictf_idf = sorted(Dictf_idf.iteritems(), key = lambda d: d[1], reverse = True) f_name = str(j + 1) + '.txt'f = open(f_name) mystr = f.read() f.close() mytitle = ""data = {}
- for i in range(mystr.__len__()) : if mystr[i] == '\n': #print mytitle res = res.__add__("\t\tarticle_titile:\"") res = res.__add__(mytitle) res = res.__add__("\"\n") res = res.__add__("\t\t[\n") for tt in range(10) : res = res.__add__("\t\t\t{\n") res = res.__add__("\t\t\t\tword:\"") res = res.__add__(Dictf_idf[tt][0]) res = res.__add__("\"\n") res = res.__add__("\t\t\t\tvalue:") res = res.__add__(str(Dictf_idf[tt][1])) res = res.__add__("\n") res = res.__add__("\t\t\t}\n") res = res.__add__("\t\t]\n")
- else: mytitle = mytitle.__add__(mystr[i]) res = res.__add__("\t]\n") res = res.__add__("}")#print res changeflie = 'tf-idf.txt'fp = open(changeflie, 'w+') fp.write(res) fp.close()
后来学习了 beautifulsoup 之后,重新实现的代码:
学习地址:
Python Data Analysis Library 或 pandas 是基于 NumPy 的一种工具,该工具是为了解决数据分析任务而创建的。Pandas 纳入了大量库和一些标准的数据模型,提供了高效地操作大型数据集所需的工具。pandas 提供了大量能使我们快速便捷地处理数据的函数和方法。你很快就会发现,它是使 Python 成为强大而高效的数据分析环境的重要因素之一。
学习地址:
-
-
-
Series 和 DataFrame 分别对应于一维的序列和二维的表结构。
pandas 约定俗成的导入方法如下:
- from pandas import Series,
- DataFrame import pandas as pd
常用函数:
- df[: 2] df["one": "three"] df.ix[: 2, :3]#前两行三列df.ix[: , "year": "year"]
- f = lambda x: x.max() - x.min() df.apply(f, axis = 0) df.apply(f, axis = 1)
- ucount = np.array(df['username'] != '-1').sum()
- subject_list = ['math', 'english', 'music'] count_list = (df[df['username'] != '-1'].loc[: , subject_list]).apply(lambda x: (x > 0).sum(), axis = 0) for pos,
- x in enumerate(count_list) : mydict[subject_list[pos]] = x
- didf = df[df['username'] != -1].groupby('sex')['id'].count() mydict['man'] = didf['man'] mydict['woman'] = didf['woman']
Echarts 是数据可视化的神器
官网地址:
下面是简单的例子:
- <!DOCTYPE html>
- <html>
- <head>
- <meta charset="utf-8">
- <title>
- ECharts
- </title>
- <!-- 引入 echarts.js -->
- <script src="echarts.min.js">
- </script>
- </head>
- <body>
- <!-- 为ECharts准备一个具备大小(宽高)的Dom -->
- <div id="main" style="width: 600px;height:400px;">
- </div>
- <script type="text/javascript">
- // 基于准备好的dom,初始化echarts实例
- var myChart = echarts.init(document.getElementById('main'));
- option = {
- title: {
- text: '到课率',
- subtext: '@monkey'
- },
- tooltip: {
- trigger: 'axis'
- },
- toolbox: {
- show: true,
- feature: {
- dataZoom: {},
- dataView: {
- readOnly: false
- },
- magicType: {
- type: ['line', 'bar']
- },
- restore: {},
- saveAsImage: {}
- }
- },
- xAxis: {
- type: 'category',
- boundaryGap: false,
- data: ['6-9', '6-10', '6-11', '6-12']
- //data: [{{ mydict['days']|join(',') }}]
- },
- yAxis: {
- type: 'value',
- axisLabel: {
- formatter: '{value} %'
- }
- },
- series: [{
- name: '到课率',
- type: 'line',
- data: [80, 85, 87, 90],
- markPoint: {
- data: [{
- type: 'max',
- name: '最大值'
- },
- {
- type: 'min',
- name: '最小值'
- }]
- },
- markLine: {
- data: [{
- type: 'average',
- name: '平均值'
- }]
- }
- },
- ]
- };
- // 使用刚指定的配置项和数据显示图表。
- myChart.setOption(option);
- </script>
- </body>
来源: http://lib.csdn.net/article/python/47325