Skip to main content
Warning: You are using the test version of PyPI. This is a pre-production deployment of Warehouse. Changes made here affect the production instance of TestPyPI (
Help us improve Python packaging - Donate today!

Utility to crawl web site looking for key words

Project Description

analyze_site is a python application to crawl a site and return a count of the keywords, provided in a file, found in the web pages of the site. The application will also return counts of the most used verb, nouns, adverbs and adjectives.

analyze_site requires Python version 3 and the following libraries:

  • nltk - Natural Language Toolkit with maxent_treebank_pos_tagger
usage: [-h] [-d DEPTH] [-r PATH_REGEX] [–verbose]
keywords_file url
positional arguments:
keywords_file Path to keywords file url URL to crawl
optional arguments:
-h, --help show this help message and exit
-d DEPTH, --depth DEPTH
 Depth to crawl
-r PATH_REGEX, --path_regex PATH_REGEX
 Regular expression to match URL
--verbose Increase logging level
Release History

Release History

This version
History Node


Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
analyze_site-0.1.3.tar.gz (4.7 kB) Copy SHA256 Checksum SHA256 Source Mar 10, 2017

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting