Skip to main content
Warning: You are using the test version of PyPI. This is a pre-production deployment of Warehouse. Changes made here affect the production instance of TestPyPI (testpypi.python.org).
Help us improve Python packaging - Donate today!

Hessian-free optimization for deep networks

Project Description

Hessian-free optimization for deep networks

Setup

Quick start

Install the package via:

pip install hessianfree

To make sure things are working, open a python interpreter and enter:

import hessianfree as hf
hf.demos.xor()

A simple xor training example will run, at the end of which it will display the target and actual outputs from the network.

Developer install

Use this if you want to track the latest changes from the repository:

git clone https://github.com/drasmuss/hessianfree.git
cd hessianfree
python setup.py develop --user

Requirements

  • python 2.7 or 3.5
  • numpy 1.9.2
  • matplotlib 1.3.1
  • optional: scipy 0.15.1, pycuda 2015.1.3, scikit-cuda 0.5.1, pytest 2.7.0

(older versions may work, but are untested)

Features

All the standard features of Hessian-free optimization from Martens (2010) and Martens and Sutskever (2011) are implemented (Gauss-Newton approximation, early termination, CG backtracking, Tikhonov damping, structural damping, etc.). In addition, the code has been designed to make it easy to customize the network you want to train, without having to modify the internal computations of the optimization process.

  • Works for feedforward and recurrent deep networks (or mixtures of the two)
  • Standard nonlinearities built in (e.g., logistic, tanh, ReLU, softmax), and support for custom nonlinearities
  • Standard loss functions (squared error, cross entropy, sparsity constraints), and support for custom loss functions
  • Various weight initialization methods (although Hessian-free optimization doesn’t usually require much tweaking)
  • Customizable connectivity between layers (e.g., skip connections)
  • Efficient implementation, taking advantage of things like activity caching
  • Optional GPU acceleration if PyCUDA and scikit-cuda are installed
  • Gradient checking (and Gauss-Newton matrix checking) implemented to help with debugging
  • Inputs can be predefined or generated dynamically by some other system (like an environmental simulation)
  • Different optimizers can be swapped out for comparison (e.g., Hessian-free versus SGD)

Documentation

View the documentation at http://pythonhosted.org/hessianfree/

In addition, examples illustrating the main features of the code can be found in demos.py.

Release History

Release History

This version
History Node

0.4.0

History Node

0.3.8

History Node

0.3.7.3

History Node

0.3.7.2

History Node

0.3.7.1

History Node

0.3.7.0

History Node

0.3.7

History Node

0.3.7.a

History Node

0.3.6

History Node

0.3.6a

History Node

0.3.5

History Node

0.3.4

History Node

0.3.4.0

History Node

0.3.4c

History Node

0.3.4b

History Node

0.3.4a

History Node

0.3.3

History Node

0.3.2

History Node

0.3.1

History Node

0.3.0

History Node

0.2.1

History Node

0.2.0

History Node

0.1.6

History Node

0.1.4

History Node

0.1.3

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
hessianfree-0.4.0.zip (64.3 kB) Copy SHA256 Checksum SHA256 Source May 26, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting