Python编程入门
目录
- 目录
- 摘要
- Python内存模型
- 文章
- virutalenv
- 常用
- 手册
- 发布管理
- 语言基础
- 项目结构
- 日志管理 – logging
- 命令行参数解析
- 配置文件解析
- python wsgi
- Python sqlAlchemy
- django
- eventlet
- stevedore
- with
- PyYAML 处理 yaml 文件
- 参考
摘要
Python相关内容。
Python内存模型
文章
python: difference between staticmethod and classmethod
virutalenv
为了避免以后由于依赖包等版本等问题导致出现各种状况，所以首先把virutalenv使用起来。
pip install virtualenv
创建虚拟环境:
virtualenv vir-env //将会创建一个名为vir-env的目录
默认会将系统中的包安装到虚拟环境中,可以使用–no-site-packages避免
virtualenv vir-env --no-site-packages
启动虚拟环境:
cd vir-env
source bin/activate
将所有依赖的python库导出：
pip freeze > requirements.txt
退出虚拟环境:
deactivate
常用
-
python第三方包的存放路径:
python -c “from distutils.sysconfig import get_python_lib; print (get_python_lib())”
-
python的import查找路径:
python -c “import sys;print sys.path”
手册
进入python命令行, 输入help()回车，进入help命令行，然后输入要查询的内容:
localhost:~ lijiao$ python
Python 2.7.10 (default, Jul 14 2015, 19:46:27)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> help()
Welcome to Python 2.7! This is the online help utility.
If this is your first time using Python, you should definitely check out
the tutorial on the Internet at http://docs.python.org/2.7/tutorial/.
Enter the name of any module, keyword, or topic to get help on writing
Python programs and using Python modules. To quit this help utility and
return to the interpreter, just type "quit".
To get a list of available modules, keywords, or topics, type "modules",
"keywords", or "topics". Each module also comes with a one-line summary
of what it does; to list the modules whose summaries contain a given word
such as "spam", type "modules spam".
help> print
发布管理
在阅读使用python的项目源码时，经常在源码顶层目录看到setup.py文件, 这个文件的作用是使用setuptools 对项目进行编译、打包、安装、卸载等操作。它的作用如同make对C语言的作用。
Setuptools is a collection of enhancements to the Python distutils (for Python 2.6 and up)
that allow developers to more easily build and distribute Python packages, especially ones
that have dependencies on other packages.
entry_point是setuptools支持的功能之一，用于将本项目中函数注册到一个系统维护到入口列表中, 第三方的代码可以直接从入口列表读取到函数，直接调用。
dynamic discovery of services and plugins
[entry_points]
console_scripts = //console_scripts比较特殊，生成的是bin/keystone-all程序，也就是控制台命
keystone-all = keystone.cmd.all:main
keystone-manage = keystone.cmd.manage:main
wsgi_scripts = //函数放入到名为wsgi_scripts的group中
keystone-wsgi-admin = keystone.server.wsgi:initialize_admin_application
keystone-wsgi-public = keystone.server.wsgi:initialize_public_application
语言基础
解析型语言。
名称空间(namespace)
总共有三个namespace:
1. Python解释器首先加载内建名称空间, 内建名称空间由`__builtins__`模块中的名字构成。
2. 然后加载模块的全局名称空间
3. 调用函数创建一个局部名称空间
作用域
TODO
项目结构
Python项目由包(package)组成, 包由模块组成(module), 模块与py文件对应。
包
包由模块和子包组成。
TopPackage/
__init__.py
SubPackage1/ #from TopPackage import SubPackage1
__init__.py
a.py #import TopPackage.SubPackage1.a
b.py
SubPackage2/
__init__.py
a.py
b.py
模块
一个模块对应一个名为”模块名.py”的文件. 一个模块可以被其它模块导入。
导入模块:
1. 导入方式：import 模块名
2. 推荐导入顺序: Python标准库模块, Python第三方模块, 应用程序自定义的模块
获取模块信息:
1. 模块的搜索路径:
通过环境变量PYTHONPATH设置。
通过sys.path查看或修改,PYTHONPATH中设置的路径会自动加载到sys.path中
2. 查看已经导入的模块:
sys.modules
日志管理 – logging
Python标准库中提供了logging包, 用于日志的管理。
def init_log(logpath,program,level):
"""
logfile: logfile will in the absolute directory.
program: program name, the logfile will be named program.log
level: the threshold for the logger
logging.CRITICAL,
logging.ERROR,
logging.WARNING
logging.INFO,
logging.DEBUG
logging.NOTSET
"""
logfile = logpath+"/"+program+".log"
handler= logging.handlers.RotatingFileHandler(logfile,'a',\
maxBytes=1024*1024*100,backupCount=5)
fmt = '%(asctime)s %(levelname)s %(filename)s %(lineno)d: %(message)s'
formatter = logging.Formatter(fmt)
handler.setFormatter(formatter)
logger = logging.getLogger(program)
logger.addHandler(handler)
logger.setLevel(level)
return logger
if __name__ == "__main__":
logger = init_log("/var/log","test",logging.DEBUG)
logger.info("starting..")
命令行参数解析
if __name__ == "__main__":
print("%d %s %s" % (sys.argv.__len__(), sys.argv[0], sys.argv[1]))
配置文件解析
python wsgi
wsgi定义了web应用与web服务器之间的接口, 使web应用可以在web服务器之间方便的移植。
wsgi只要求web app具有一个__call__方法:
1. `__call__`方法必须接收两个位置参数:
一个包含类似于cgi变量的字典;
一个用于向web server发送http状态和头信息的回调函数。
2. `__call__`方法必须返回一个字符串列表, 该列表被当作回应的消息体。
python paste
python的paste库提供了管理wsgi接口的方式。通过paste的配置文件, 控制wsgi接口之间关系。
paste配置文件以Section为单位, 每个Section的类型可以是:
app、composite app、filter、pipeline、filter-app
每个section的配置格式相同:
[section type: name]
use=XXX # 或者 paste.XXX=
other1=xxx #其它的配置
other2=xxx
app
app是最后承当实际工作的。
接口约定:
def __call__(self, env, start_response):
paste.app_factory的约定:
def factory(cls, global_conf, **local_conf):
# 返回满足app接口约定的app
# global_conf是全局配置
# local_conf是当前section中的配置
use项内容:
[app: appname]
use = XXX
# config:another_config_file.ini#app_name -->转向另一个app的配置
# egg:MyApp --> 调用egg包中的处理接口
# call: my.project:myapplication --> 直接指定处理接口的路径
# myotherapp --> 跳转到另一个app
采用factory:
[app: appname]
paste.app_factory = test_wsgi:TestApp.factory
可以在section的配置中通过set改写全局配置，例如:
[DEFAULT]
name = test
version = 1.0
[app:test1]
paste.app_factory = test_wsgi:TestApp.factory
version = 2.0
set name = test1 #被修改为test1
Composite App
多个app可以组合在一起，通过url进行区分。
[composite:main]
use = egg:Paste#urlmap #通过url进行map
/ = mainapp
/files = staticapp
[app:mainapp]
use = egg:MyApp
[app:staticapp]
use = egg:Paste#static
document_root = /path/to/docroot
paste.composite_factory约定:
def composite_factory(loader, global_config, **local_conf):
return wsgi_app
The loader argument is an object that has a couple interesting methods.
get_app(name_or_uri, global_conf=None) return a WSGI application with the
given name.
get_filter and get_server work the same way.
Filter
Filters are callables that take a WSGI application as the only argument, and return a “filtered” version of that application
[filter: filtername]
# use or factory
paste.filter_factory的约定:
def filter_factory(global_conf, req_usernames):
req_usernames = req_usernames.split()
def filter(app):
return AuthFilter(app, req_usernames)
return filter
Filter app
The [filter-app:blog] section means that you want an application with a filter applied. The application being filtered is indicated with next (which refers to the next section)
[filter-app:blog]
use = egg:Authentication#auth
next = blogapp
roles = admin
htpasswd = /home/me/users.htpasswd
[app:blogapp]
use = egg:BlogApp
database = sqlite:/home/me/blog.db
Pipeline
pipeline: is used when you need apply a number of filters. It takes one configuration key pipeline (plus any global configuration overrides you want).
pipeline is a list of filters ended by an application, like:
[pipeline:main]
pipeline = filter1 egg:FilterEgg#filter2 filter3 app
[filter:filter1]
...
示例1
test_paste.ini中定义了一个名为test1的app
[DEFAULT]
name = test
version = 1.0
[app:test1]
paste.app_factory = test_wsgi:TestApp.factory
version = 2.0
set name = test1 #该写了全局配置
test_paste.py中给出了Test的实现
import os
import eventlet
from eventlet import wsgi, listen
from paste import deploy
class TestApp(object):
@classmethod
def factory(cls, global_conf, **local_conf):
return cls()
def __call__(self, env, start_response):
print env
start_response("200 OK",())
return ["welcome"]
if __name__ == '__main__':
_path = os.path.dirname(os.path.abspath(__file__))
f = "config:%s" % os.path.join(_path,"test_paste.ini")
host = "0.0.0.0"
port = 8080
appname="test1" # app name
_socket = listen((host,port))
app = deploy.loadapp(f, appname)
server = eventlet.spawn(wsgi.server, _socket, app)
server.wait()
示例2 - Filter
import os
import eventlet
from eventlet import wsgi, listen
from paste import deploy
class UrlFilter(object):
@classmethod
def factory(cls, global_conf, **local_conf):
def filter(app):
return UrlFilter(app, global_conf)
return filter
def __init__(self, app, global_conf):
self.app = app
self.global_conf = global_conf
def __call__(self, environ, start_response):
#TODO: You can check something at here
print 'Url Filter'
return self.app(environ,start_response) ##Filter检查通过, 进入下一个app
class AuthFilter(object):
@classmethod
def factory(cls, global_conf, **local_conf):
def filter(app):
return AuthFilter(app, global_conf)
return filter
def __init__(self, app, global_conf):
self.app = app
self.global_conf = global_conf
def __call__(self, environ, start_response):
#TODO: You can check something at here
print 'Auth Filter'
return self.app(environ,start_response)
class WelcomeApp(object):
@classmethod
def factory(cls, global_conf, **local_conf):
return cls()
def __init__ (self, *args, **kwargs):
pass
def __call__(self, env, start_response):
print 'WelcomeApp Filter'
start_response('200 OK',())
return ['welcome']
if __name__ == '__main__':
_path = os.path.dirname(os.path.abspath(__file__))
f = 'config:%s' % os.path.join(_path,'test_paste.ini')
host = '0.0.0.0'
port2 = 8080
appname2='admin' # app name
_socket2 = listen((host,port2))
app2 = deploy.loadapp(f, appname2)
server2 = eventlet.spawn(wsgi.server, _socket2, app2)
server2.wait()
示例3 - Openstack: Keystone
[filter:sizelimit]
paste.filter_factory = keystone.middleware:RequestBodySizeLimiter.factory
[pipeline:api_v3]
# The last item in this pipeline must be service_v3 or an equivalent
# application. It cannot be a filter.
pipeline = sizelimit url_normalize build_auth_context token_auth admin_token_auth \
xml_body_v3 json_body ec2_extension_v3 s3_extension simple_cert_extension \
revoke_extension service_v3
[composite:main]
use = egg:Paste#urlmap
/v2.0 = public_api
/v3 = api_v3
/ = public_version_api
Python sqlAlchemy
#coding=utf-8
import sqlalchemy
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import create_engine
from sqlalchemy import Column,Integer,String,ForeignKey
from sqlalchemy.orm import sessionmaker,relationship,backref
Base = declarative_base()
class User(Base): #表users
__tablename__ = 'users' #指定表明
#创建对应的列
id = Column(Integer, primary_key=True)
name = Column(String(50)) #有的数据库需要设置string长度,如mysql
fullname = Column(String(50))
password = Column(String(50))
def __repr__(self):
return "<User(name='%s',fullname='%s',password='%s')>"%(
self.name, self.fullname, self.password)
class Address(Base):
__tablename__ ='addresses'
id = Column(Integer, primary_key=True)
email_address = Column(String(50), nullable=False)
#外键:索引到users.id
user_id = Column(Integer, ForeignKey('users.id'))
#在Address中增加user属性，该属性的值是address的外键对应的user表中中记录
#同时在User中增加addresses属性，该属性的值是user对应的addresses表中的记录
user = relationship("User", backref=backref('addresses', order_by=id))
def __repr__(self):
return "<Addresses(email_address='%s')>" % self.email_address
#连接到数据库
#engine = create_engine('sqlite:///:memory:',echo=False)
engine = create_engine('mysql://root:@localhost/sqlalchemy',echo=False)
#创建数据库表
Base.metadata.create_all(engine)
#获取一个数据库会话
Session = sessionmaker(bind=engine)
session = Session()
#生成一条记录
user_a = User(name='a',fullname='aaaaaaaaa',password='adfadfdafadf')
session.add(user_a) #现在user_a缓存在session中，还没有提交到数据库中
session.commit() #将session中更新提交到数据库中
#如果不手动提交，更新会在执行查询操作时自动提交
#查询，验证记录已经提交到数据库
rows=session.query(User).all()
print rows
#修改记录的值
user_a.password='1234aaaaaaaaaaaaa'
session.commit() #将session中更新提交到数据库中
#查询, 验证记录的修改已经提交到数据库
rows=session.query(User).all()
print rows
#在查询结果中修改记录的值
for row in rows:
row.name='bbbbbbaaaaa'
session.commit()
#查询，验证在查询结果中的修改是否提交到数据库
rows=session.query(User).all()
print rows
addr1 = Address(email_address="[email protected]")
addr2 = Address(email_address="[email protected]")
user_a.addresses=[addr1, addr2]
user_b = User(name='b',fullname='bbbb',password='adfadfdafadf')
user_b.addresses=[addr2] #addr2的外键会被自动修改为user_b的id
session.commit()
#查询，验证addresses中的记录
rows = session.query(Address).all()
print rows
django
安装:
pip install django
查看安装结果:
$ python
Python 2.7.10 (default, Jul 14 2015, 19:46:27)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
>>> print(django.get_version())
1.9.4
创建项目：
django-admin startproject class
创建应用：
python ./manage.py startapp wangyi
在项目目录的setting.py中添加应用wangyi：
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'wangyi',
]
在setting.py中配置数据库地址：
DATABASES = {
# 'default': {
# 'ENGINE': 'django.db.backends.sqlite3',
# 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
# }
'default': {
'ENGINE': 'django.db.backends.mysql',
'HOST': '127.0.0.1',
'NAME': 'spider',
'USER': 'spider',
'PASSWORD': '123456',
'OPTIONS':{
'charset': 'utf8mb4',
},
}
}
在应用的wangyi/models.py中太添加和数据库表对应的类：
# Create your models here.
class WangYiClass(models.Model):
product_id = models.IntegerField(db_index=True)
price = models.IntegerField()
discount_price = models.IntegerField()
name = models.CharField(max_length=128)
description = models.TextField()
picture_url = models.CharField(max_length=256)
record_time = models.DateTimeField(auto_now=True)
然后创建数据库：
python manage.py makemigrations
python manage.py migrate
django的models的使用见Models，里面给出了可以使用的Field Options。
启动server：
python manage.py runserver
在命令行中引用django的代码：
import os
import sys
import django
# load django environment
sys.path.append(os.path.dirname(os.path.dirname(os.path.realpath(__file__))))
os.environ['DJANGO_SETTINGS_MODULE'] = 'banyung.settings'
django.setup()
from wangyi.models import WangYiClass
if __name__ == '__main__':
_,created = WangYiClass.objects.get_or_create(
product_id="",
price="",
discount_price="",
name="",
description="",
picture_url="",
)
eventlet
eventlet是python的协程库。
stevedore
stevedore是一个插件加载库。
https://pypi.python.org/pypi/stevedore/
with
https://docs.python.org/3/reference/compound_stmts.html#the-with-statement
使用with时，with的作用对象必须是实现了属性__enter__(self)和__exit__(self, exc_type, exc_value, traceback)
示例:
class Sample:
def __init__(self,name):
self.name=name;
def __enter__(self):
print("__enter__")
return self
def __exit__(self, type, value, trace):
print("__exit__")
with Sample("hahah") as s:
print("sample name%s", s.name)
执行步骤:
1 类Sample的__init__函数被执行, 构建了对象
2 对象的__enter__函数被执行, 返回的值被赋予了"with XXX as s"中的s
3 with下的语句块被执行
4 对象的__exit__函数被执行 //__exit__函数用来处理with下语句块执行时抛出的异常
PyYAML 处理 yaml 文件
pip install PyYAML
import yaml
import os
data = {
"title": "this is title",
"list": ['item1','item2','item3'],
"zh": "中文",
"map": {
"sub-map1": "sub-map1",
"sub-map2": "sub-map2",
}
}
print dump(data,default_flow_style=False,allow_unicode=True)
输出结果如下：
list:
- item1
- item2
- item3
map:
sub-map1: sub-map1
sub-map2: sub-map2
title: this is title
zh: !!python/str '中文'
注意如果没有将设置参数default_flow_style=False，输出的会是下面的样式：
list: [item1, item2, item3]
map: {sub-map1: sub-map1, sub-map2: sub-map2}
title: this is title
zh: !!python/str '中文'
参考
