pip.installed with local tar.gz file

[daks]

Description of Issue/Question

My servers have a limited access to internet, so I want to pip install a package using a local tar.gz file.
This tar.gz file works well when installing it directly with pip

pip install file:///collectd-ntp-0.0.4.tar.gz

I have searched in documentation and issues and do not find any clear statement that local files are supported in pip.installed state.

Setup

Saltstack version 2016.11.8+ds1 (some servers run .5), either running in master-minion setup or masterless (using salt-call in test-kitchen environment)

My original SLS file looks (I removed require/require_in and other ID in file)

# collectd/ntpd.sls
collectd-ntp-module:
  pip.installed:
  - name: 'collectd-ntp'
  - source: salt://collectd/files/collectd-ntp-0.0.4.tar.gz

collectd-ntp-0.0.4.tar.gz is in directory collectd/files

Steps to Reproduce Issue

In masterless environnement, I run

salt-call --local state.sls_id collectd-ntp-module collectd.ntpd

I always have an error about the impossibility to reach https://pypi.python.org

          ID: collectd-ntp-module
    Function: pip.installed
        Name: collectd-ntp
      Result: False
     Comment: Failed to install packages: collectd-ntp. Error: Downloading/unpacking collectd-ntp
                Cannot fetch index base URL https://pypi.python.org/simple/
                Could not find any downloads that satisfy the requirement collectd-ntp
              Cleaning up...
              No distributions at all found for collectd-ntp
              Storing debug log for failure in /root/.pip/pip.log

Versions Report

$ salt-call --versions-report
Salt Version:
           Salt: 2016.11.8
 
Dependency Versions:
           cffi: 0.8.6
       cherrypy: Not Installed
       dateutil: 2.2
      docker-py: Not Installed
          gitdb: 0.5.4
      gitpython: 0.3.2 RC1
          ioflo: Not Installed
         Jinja2: 2.9.4
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.2
   mysql-python: 1.2.3
      pycparser: 2.10
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.9 (default, Jun 29 2016, 13:08:31)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 14.4.0
           RAET: Not Installed
          smmap: 0.8.2
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5
 
System Versions:
           dist: debian 8.9 
        machine: x86_64
        release: 3.16.0-4-amd64
         system: Linux
        version: debian 8.9 

[daks]

For information, I also tested to file.managed in /tmp my tar.gz and then pip.install from this /tmp/file.tar.gz, without success.

[Ch3LL]

I'm not seeing that as an option, but in the code it looks like it just runs the executable pip install so i'm surprised your file.managed and then pip install did not work. What did that state look like?

Also for now i'll approve this as a feature request.

[daks]

Complete sls file is, it comes from saltstack-formulas/collectd-formula

{% from "collectd/map.jinja" import collectd_settings with context %}

include:
  - collectd
  - collectd.python

collectd-ntpd-wheel:
  file.managed:
    - name: /tmp/collectd-ntp-0.0.4.tar.gz
    - source: salt://collectd/files/collectd-ntp-0.0.4.tar.gz

collectd-ntpd-module:
  pip.installed:
  - name: 'collectd-ntp'
  - source: /tmp/collectd-ntp-0.0.4.tar.gz
  - require_in:
    - service: collectd
    - file: collectd-ntpd-wheel
  - watch_in:
    - service: collectd

{{ collectd_settings.plugindirconfig }}/ntpd.conf:
  file.managed:
    - source: salt://collectd/files/ntpd.conf
    - user: {{ collectd_settings.user }}
    - group: {{ collectd_settings.group }}
    - mode: 644
    - template: jinja
    - watch_in:
      - service: collectd-service

file.manage state ID collectd-ntpd-wheel works well

$ salt-call [...] state.sls_id collectd-ntpd-wheel collectd.ntpd

...
    Function: file.managed
    Name: /tmp/collectd-ntp-0.0.4.tar.gz
    Result: True
...
$ ls /tmp/collectd-ntp-0.0.4.tar.gz 
/tmp/collectd-ntp-0.0.4.tar.gz

but not pip.installed collectd-ntpd-module

$ salt-call [...] state.sls_id collectd-ntpd-module collectd.ntpd

    Function: pip.installed
    Name: collectd-ntp
    Result: False
    Comment: Failed to install packages: collectd-ntp. Error: Downloading/unpacking collectd-ntp
        Cannot fetch index base URL https://pypi.python.org/simple/
        Could not find any downloads that satisfy the requirement collectd-ntp
        Cleaning up...
        No distributions at all found for collectd-ntp
        Storing debug log for failure in /root/.pip/pip.log

[daks]

We have found a way to make it work using a temporary file.managed step. The solution is

collectd-ntpd-module:
  pip.installed:
    - name: file:///tmp/collectd-ntp-0.0.4.tar.gz
...

Using source to specify local file does not work, but giving it to name does.

I think Improving documentation of pip.installed state would be interesting, to explicity indicate how it works.

And adding the possibility to use a source with salt:// fileserver would be really great, like you can in other states.
@Ch3LL Do you want me to split the issue (or rewrite subject) between these two points?

[Ch3LL]

Yeah lets keep this issue open for adding the source salt:// option and whenever someone adds that option we will make sure it includes docs for this new option as well :) So we can just use this issue thats fine

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

[daks]

This feature request is still valid.

[stale[bot]]

Thank you for updating this issue. It is no longer marked as stale.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

[daks]

still relevant

[stale[bot]]

Thank you for updating this issue. It is no longer marked as stale.