Skip to content

Releases: scrapy/protego

0.6.0

29 Jan 10:57

Choose a tag to compare

  • Added official support for Python 3.14.
  • Dropped support for Python 3.9 and PyPy 3.10.
  • CI fixes and improvements.

0.5.0

24 Jun 13:59

Choose a tag to compare

  • Restructured the code, splitting the single protego.py file into
    multiple modules. The public API remains the same but some internal names
    may now be available at different import paths.
  • Added type hints and py.typed.
  • Added official support for PyPy 3.11.
  • Switched the build backend from setuptools to hatchling.
  • Switched from setup.py to pyproject.toml.
  • CI fixes and improvements.

0.4.0

24 Jun 13:59
ecc22d4

Choose a tag to compare

  • Dropped Python 3.8 support, added official Python 3.13 support.
  • Added support for //-prefixed URLs.
  • Improved the rendering of the README.rst code snippets in GitHub.

0.3.1

05 Apr 10:11

Choose a tag to compare

  • Added official support for Python 3.12.

  • = is no longer percent-encoded in patterns, fixing many scenarios where
    patterns included query strings.

0.3.0

09 Aug 17:42

Choose a tag to compare

  • Changed requirements:

    • Dropped support for Python 2.7, 3.5, 3.6, and 3.7, and added support
      for 3.11 and for the upcoming 3.12.

    • six is no longer a dependency.

  • Added support for the Visit-Time directive.

  • Fixed leading asterisks in allow and disallow values not being properly
    interpreted.

  • Protego.parse() now raises value error when content is not a string.

Protego 0.2.1

19 Feb 14:19
aed7040

Choose a tag to compare

  1. Fixes incorrect readme content-type specified in setup.py (#21)

Protego 0.2.0

12 Feb 09:42

Choose a tag to compare

  1. Fixes disallow not working when no path is provided (#19)
  2. Adds support to handle sitemaps that come before any User-Agent directives (#15)
  3. Makes smaller wheel distributions (#10)
  4. Adds code coverage support (#7)
  5. Adds test coverage for Python versions 3.8, 3.9 and 3.10 (#20).
  6. Code cleanups (#9, #11)

Protego 0.1.16

09 Dec 12:26

Choose a tag to compare

Protego 0.1.16 fixes (#8) an issue with robots.txt files containing absolute URLs as values for allow and disallow directives, where their path would be interpreted ignoring their protocol and netloc, leading to unexpected results (#4). Now, values of allow and disallow directives are always interpreted as if they started with a /, as Google’s specification dictates that they must always start with /.

We’ve also improved (#3) our test coverage for malformed disallow directives.

Initial release of Protego

28 Aug 15:37

Choose a tag to compare

0.1.7

changes travis config

Initial release of Protego

20 Aug 14:36
452364a

Choose a tag to compare

0.1.3

Changes Travis config