Skip to content

[SPARK-30665][DOCS][BUILD][PYTHON] Eliminate pypandoc dependency #27376

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
nchammas wants to merge 9 commits into from
Closed

[SPARK-30665][DOCS][BUILD][PYTHON] Eliminate pypandoc dependency #27376

Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
4098acf
remove pypandoc
nchammas Jan 28, 2020
25c3847
remove pypandoc from Dockerfile and run-pip-tests
nchammas Jan 29, 2020
ce218de
remove pypandoc from docs/dev requirements
nchammas Jan 29, 2020
16d42c3
we need numpy to build the PySpark ML docs!
nchammas Jan 29, 2020
70f3b88
Merge branch 'master' of https://github.com/apache/spark into SPARK-3…
nchammas Jan 29, 2020
6fa5b8c
remove numpy from here
nchammas Jan 30, 2020
dd77815
Merge branch 'master' into SPARK-30665-pypandoc
nchammas Jan 30, 2020
71182fe
call out pandoc dependency
nchammas Jan 30, 2020
98e0a31
separate R prerequisites from general API prereqs
nchammas Jan 30, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 3 additions & 10 deletions python/setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,21 +153,15 @@ def _supports_symlinks():
# will search for SPARK_HOME with Python.
scripts.append("pyspark/find_spark_home.py")

# Parse the README markdown file into rst for PyPI
long_description = "!!!!! missing pandoc do not upload to PyPI !!!!"
try:
import pypandoc
long_description = pypandoc.convert('README.md', 'rst')
except ImportError:
print("Could not import pypandoc - required to package PySpark", file=sys.stderr)
except OSError:
print("Could not convert - pandoc is not installed", file=sys.stderr)
with open('README.md') as f:
long_description = f.read()

setup(
name='pyspark',
version=VERSION,
description='Apache Spark Python API',
long_description=long_description,
long_description_content_type="text/markdown",
author='Spark Developers',
author_email='[email protected]',
url='https://github.com/apache/spark/tree/master/python',
Expand Down Expand Up @@ -213,7 +207,6 @@ def _supports_symlinks():
scripts=scripts,
license='http://www.apache.org/licenses/LICENSE-2.0',
install_requires=['py4j==0.10.8.1'],
setup_requires=['pypandoc'],
extras_require={
'ml': ['numpy>=1.7'],
'mllib': ['numpy>=1.7'],
Expand Down