A-A+

ubuntu14.04安装Scrapy框架

2016年04月02日 运维 评论 2 条 阅读 96 次

ubuntu14.04安装Scrapy框架,首先,安装scrapy 需要以下程序或者库:

  • python 2.7
  • lxml
  • OpenSSL
  • pip or easy_install

一、pip是作为python包索引器easy_install的替代品,用于安装和管理Python包。

sudo apt-get install python-pip

二、我们必须要用下面的命令安装python开发库。如果包没有安装那么就会在安装scrapy框架的时候报关于python.h头文件的错误。

sudo apt-get install python-dev

三、验证是否安装Python 2.7 和 lxml 和 OpenSSL 和 pip

1、验证Python 2.7 : python -V

2、验证lxml和OpenSSL:

root@hoster:~# python
Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import lxml
>>> import OpenSSL

3、验证pip:

root@hoster:~# pip

Usage:   
  pip <command> [options]

Commands:
......

注:如何安装lxml?:

1、sudo apt-get update

2、sudo apt-get install libxml2-dev libxslt-dev python-dev

3、sudo apt-get build-dep python-lxml(依赖包)

4、pip install lxml

四、scrapy框架既可从deb包安装也可以从源码安装。我们用pip(Python 包管理器)安装了deb包了。

sudo pip install scrapy(注:如果安装错误,请执行以下命令,安装依赖环境 sudo apt-get install libssl-dev libffi-dev,sudo pip install pyopenssl)

五、验证是否安装scrapy成功:

root@hoster:~/testing# scrapy
Scrapy 1.0.5 - project: testing

Usage:
  scrapy <command> [options] [args]

Available commands:
  bench         Run quick benchmark test
  check         Check spider contracts
  commands      
  crawl         Run a spider
  edit          Edit spider
  fetch         Fetch a URL using the Scrapy downloader
  genspider     Generate new spider using pre-defined templates
  list          List available spiders
  parse         Parse URL (using its spider) and print the results
  runspider     Run a self-contained spider (without creating a project)
  settings      Get settings values
  shell         Interactive scraping console
  startproject  Create new project
  version       Print Scrapy version
  view          Open URL in browser, as seen by Scrapy

Use "scrapy <command> -h" to see more info about a command

Copyright:www.cplusplus.me Share、Open- C/C++程序员之家

2 条留言  访客:0 条  博主:1 条   引用: 1 条

  1. C/C++程序员之家

    如果安装(pip install scrapy)错误,请执行以下命令,安装依赖环境
    sudo apt-get install libssl-dev libffi-dev

来自外部的引用: 1 条

  • centos安装scrapy – C/C++程序员之家

给我留言