[v6] python: create installable package

[PATCH v6 07/25] python: add MANIFEST.in

Posted by John Snow 4 years, 9 months ago

When creating a source distribution via 'python3 setup.py sdist', the
VERSION and PACKAGE.rst files aren't bundled by default. Create a
MANIFEST.in file that instructs the build tools to include these so that
installation from source dists won't fail.

(This invocation is required by 'tox', as well as by the tooling needed
to upload packages to PyPI.)

Signed-off-by: John Snow <jsnow@redhat.com>
---
 python/README.rst  | 2 ++
 python/MANIFEST.in | 2 ++
 2 files changed, 4 insertions(+)
 create mode 100644 python/MANIFEST.in

diff --git a/python/README.rst b/python/README.rst
index 7a0dc5dff4a..86364367261 100644
--- a/python/README.rst
+++ b/python/README.rst
@@ -33,6 +33,8 @@ Files in this directory
 -----------------------
 
 - ``qemu/`` Python package source directory.
+- ``MANIFEST.in`` is read by python setuptools, it specifies additional files
+  that should be included by a source distribution.
 - ``PACKAGE.rst`` is used as the README file that is visible on PyPI.org.
 - ``README.rst`` you are here!
 - ``VERSION`` contains the PEP-440 compliant version used to describe
diff --git a/python/MANIFEST.in b/python/MANIFEST.in
new file mode 100644
index 00000000000..2b1ee8b4e72
--- /dev/null
+++ b/python/MANIFEST.in
@@ -0,0 +1,2 @@
+include VERSION
+include PACKAGE.rst
\ No newline at end of file
-- 
2.30.2

Re: [PATCH v6 07/25] python: add MANIFEST.in

Posted by Cleber Rosa 4 years, 8 months ago

On Wed, May 12, 2021 at 07:12:23PM -0400, John Snow wrote:
> When creating a source distribution via 'python3 setup.py sdist', the
> VERSION and PACKAGE.rst files aren't bundled by default. Create a
> MANIFEST.in file that instructs the build tools to include these so that
> installation from source dists won't fail.
> 
> (This invocation is required by 'tox', as well as by the tooling needed
> to upload packages to PyPI.)
> 
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>  python/README.rst  | 2 ++
>  python/MANIFEST.in | 2 ++
>  2 files changed, 4 insertions(+)
>  create mode 100644 python/MANIFEST.in
>

I was about to propose mypy.ini to be included here, but given
that it's merged into setup.cfg later in this series:

Reviewed-by: Cleber Rosa <crosa@redhat.com>

---

Note to self (and to you) when generating the sdist, I get:

   ...
   package init file 'qemu/__init__.py' not found (or not a regular file)
   package init file 'dist/__init__.py' not found (or not a regular file)
   ...

Which may not be too harmful, but deserves investigation.

Re: [PATCH v6 07/25] python: add MANIFEST.in

Posted by John Snow 4 years, 8 months ago

On 5/24/21 10:42 PM, Cleber Rosa wrote:
> On Wed, May 12, 2021 at 07:12:23PM -0400, John Snow wrote:
>> When creating a source distribution via 'python3 setup.py sdist', the
>> VERSION and PACKAGE.rst files aren't bundled by default. Create a
>> MANIFEST.in file that instructs the build tools to include these so that
>> installation from source dists won't fail.
>>
>> (This invocation is required by 'tox', as well as by the tooling needed
>> to upload packages to PyPI.)
>>
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>>   python/README.rst  | 2 ++
>>   python/MANIFEST.in | 2 ++
>>   2 files changed, 4 insertions(+)
>>   create mode 100644 python/MANIFEST.in
>>
> 
> I was about to propose mypy.ini to be included here, but given
> that it's merged into setup.cfg later in this series:
> 
> Reviewed-by: Cleber Rosa <crosa@redhat.com>
> 
> ---
> 
> Note to self (and to you) when generating the sdist, I get:
> 
>     ...
>     package init file 'qemu/__init__.py' not found (or not a regular file)

I think this one is fine, it's just a consequence of PEP420.

>     package init file 'dist/__init__.py' not found (or not a regular file)

This is kinda weird, but it only happens if you run sdist a *second* 
time, and it has created a dist/ folder. It shouldn't be trying to 
package its own dist folder ... !

After a 60 second look I don't know how to rectify this behavior, I will 
look into it.

>     ...
> 
> Which may not be too harmful, but deserves investigation.
> 

fwiw clean run vs re-run:

--- run1	2021-05-25 10:10:37.166975879 -0400
+++ run2	2021-05-25 10:10:48.527966676 -0400
@@ -1,11 +1,10 @@
  running sdist
  running egg_info
-creating qemu.egg-info
  writing qemu.egg-info/PKG-INFO
  writing dependency_links to qemu.egg-info/dependency_links.txt
  writing top-level names to qemu.egg-info/top_level.txt
-writing manifest file 'qemu.egg-info/SOURCES.txt'
  package init file 'qemu/__init__.py' not found (or not a regular file)
+package init file 'dist/__init__.py' not found (or not a regular file)
  reading manifest file 'qemu.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  writing manifest file 'qemu.egg-info/SOURCES.txt'
@@ -35,6 +34,5 @@
  copying qemu/utils/__init__.py -> qemu-0.6.1.0a1/qemu/utils
  copying qemu/utils/accel.py -> qemu-0.6.1.0a1/qemu/utils
  Writing qemu-0.6.1.0a1/setup.cfg
-creating dist
  Creating tar archive
  removing 'qemu-0.6.1.0a1' (and everything under it)



at the very least, it doesn't look like it does anything actively 
harmful, but it seems to want to find sources in the dist, so I should 
steer it towards exclusively the qemu/ folder.

Thanks for this, Cleber!

--js