Scripts are stuck at "running state" on execution

[Ariteme]

I am currently trying to start creating scripts for netbox, but on every run job stuck at "running" state and never quits.
Here is the log from netbox-worker-docker:

08:57:32 default: extras.scripts.run_script(commit=False, data={}, job=<Job: 1aa8a2f4-30dc-4d71-b6b2-e6b864181efb>, request=<utilities.request.NetBoxFakeRequest object at 0x7f1716bb1950>) (1aa8a2f4-30dc-4d71-b6b2-e6b864181efb)
08:57:32 [Job 1aa8a2f4-30dc-4d71-b6b2-e6b864181efb]: exception raised while executing (extras.scripts.run_script)
Traceback (most recent call last):
File "/opt/netbox/venv/lib/python3.11/site-packages/rq/worker.py", line 1430, in perform_job
rv = job.perform()
^^^^^^^^^^^^^
File "/opt/netbox/venv/lib/python3.11/site-packages/rq/job.py", line 1280, in perform
self._result = self._execute()
^^^^^^^^^^^^^^^
File "/opt/netbox/venv/lib/python3.11/site-packages/rq/job.py", line 1317, in _execute
result = self.func(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/netbox/netbox/extras/scripts.py", line 627, in run_script
script = ScriptModel.objects.get(pk=job.object_id).python_class()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable

P.S. Just to mention that it just does not run me everything under Class, but is able to run regular python command. For example I'm able to do ping via script, but it shouldn't be under "run()" function to be executed.

Kindly asking for assist.

[candlerb]

Netbox version? Can you show the script?

I suggest you start with a simple "hello world" type of script, make sure you can run it, and then this would prove that the issue is with your script.

There are some sample scripts here, although many may not have been updated for Netbox 4.0.

[Ariteme]

8e461e56f6e1 (v4.0.2)
I tried example from the list you provided: add_device_type_components.py. It has the same issue.
In additional here is my script you asked for:

#!/opt/netbox/venv/bin/python
import django, os, sys
sys.path.append('/opt/netbox/netbox')
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'netbox.settings')
django.setup()

import logging
from extras.scripts import Script
import subprocess

# Configure logging
logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    handlers=[
        logging.FileHandler('/opt/netbox/netbox/scripts/ping_device.log'),
        logging.StreamHandler()
    ]
)
logger = logging.getLogger(__name__)

class PingDevice(Script):
    class Meta:
        name = "Ping Device"
        description = "Ping a device with IP 10.60.60.17"
        default_commit = False
        logger.info("Logger before RUN")

    def run(self):
        ip_address = "10.60.60.17"
        logger.info(f"Starting ping to {ip_address}")

        try:
            # Use the ping command
            result = subprocess.run(
                ["ping", "-c", "4", ip_address],
                stdout=subprocess.PIPE,
                stderr=subprocess.PIPE,
                text=True,
                timeout=60  # Add a timeout (in seconds) to prevent the script from hanging indefinitely
            )

            if result.returncode == 0:
                logger.info(f"Ping to {ip_address} successful:\n{result.stdout}")
                self.log_success(f"Ping to {ip_address} successful:\n{result.stdout}")
            else:
                logger.error(f"Ping to {ip_address} failed:\n{result.stderr}")
                self.log_failure(f"Ping to {ip_address} failed:\n{result.stderr}")

        except subprocess.TimeoutExpired:
            logger.error(f"Ping to {ip_address} timed out")
            self.log_failure(f"Ping to {ip_address} timed out")

        except Exception as e:
            logger.error(f"An error occurred: {e}")
            self.log_failure(f"An error occurred: {e}")
logger.info("Logger after RUN")

Here is the docker stack:
NAMES
docker-netbox-worker-1
docker-netbox-housekeeping-1
docker-netbox-1
docker-redis-1
docker-postgres-1
docker-redis-cache-1

The container that the script appear in after adding via GUI is docker-netbox-1.
The one that is throwing me an error mentioned before is docker-netbox-worker-1.
From other thread I understood that there might be an issue with the django that can't find a file referenced, but I am not sure how exactly script was initialized if I run it via GUI Customization>Scripts.

UPD:
Tried hello world

from extras.scripts import Script


class AddDeviceTypeComponents(Script):
    class Meta:
        name = "Hello world!"
        description = "Hello world!"

    def run(self, data, commit):
        print("Hello world!")

Having the same issue with the worker.

[Ariteme]

I edited my comment above to better visibility.
Tried the code you suggested, it still doesn't run it to me.

Here are the screenshot of the flow:

1. Uploading your suggested code via webGUI
   
   It stored me the file in container "docker-netbox-1" in path "/opt/netbox/netbox/scripts/example_netbox.py"
   in container "docker-netbox-worker-1" there is no such file.
   Then I run the script by "Run Script" button in WebGUI.
   It stuck in "running state"
   
   and the only strange log I receive on my cluster is from container "docker-netbox-worker-1" where we can see the error mentioned above. Here is the screenshot:
   

Seems like it does not matter which netbox script it runs it just stuck on it with the python error.

[candlerb]

Then there's something different between your system and mine. I'm running Netbox standalone (without docker) which is one possible difference.

[candlerb]

Additional info: while re-uploading my existing scripts, I was able to replicate a problem similar to yours when deleting an existing script:

This is weird. The script has been deleted from the filesystem, but remains in the database:

netbox4=# select id,file_root,file_path from core_managedfile where file_path like '%showdata%';
 id | file_root |  file_path
----+-----------+-------------
 32 | scripts   | showdata.py
(1 row)

(Aside: historically, Netbox has been very poor about dealing with scripts that have problems in them, which makes them very hard to develop. It's not just you or me)

It turns out that trying to delete the event rule by itself gave the same exception.

I had to delete the eventrule by hand using SQL:

netbox4=# delete from extras_eventrule where name='test-showdata';
ERROR:  update or delete on table "extras_eventrule" violates foreign key constraint "extras_eventrule_con_eventrule_id_c177fa54_fk_extras_ev" on table "extras_eventrule_object_types"
DETAIL:  Key (id)=(2) is still referenced from table "extras_eventrule_object_types".
netbox4=# delete from extras_eventrule_object_types where eventrule_id=2;
DELETE 1
netbox4=# delete from extras_eventrule where name='test-showdata';
DELETE 1

[Ariteme]

Yeah, I am deleting the scripts via GUI without any problem.
The problem occurs when I am trying to run any of it.

[jirivrany]

Hello, I had similar issue. The problem was in configuration of docker volumes and their mapping into the netbox and netbox-worker containers.
What you can try:

1. check your config.py if there is SCRIPTS_ROOT set. If so, be sure to map your volumes to this. Or disable this directive, and keep it on default /opt/netbox/netbox/scripts
2. check your docker-compose.yml and create your scripts dirs as the volumes are described:

 volumes:
    - ./configuration:/etc/netbox/config:z,ro
    - netbox-media-files:/opt/netbox/netbox/media:rw
    - netbox-reports-files:/opt/netbox/netbox/reports:rw
    - netbox-scripts-files:/opt/netbox/netbox/scripts:rw

3. if you changing the volumes in docker-compose-override.yml be sure, that you map the scripts to the netbox-worker too. This was my solution. I'm using override a lot, because I need to work on plugiin development localy. So here is my complete override file:

x-netbox-common: &common
    image: netbox:407-plugins
    build:
      context: .
      dockerfile: Dockerfile-Plugins
    restart: unless-stopped
    user: "0:0" # root user, without it setup.py develop will not work
    tty: true
    env_file:
      - env/netbox.env
      

services:
  netbox:
    <<: *common
    container_name: netbox
    command: >
      sh -c "/opt/netbox/venv/bin/pip install -e /opt/netbox/netbox/netbox_cesnet_services_plugin/ &&
      /opt/netbox/venv/bin/python manage.py makemigrations &&
      /opt/netbox/venv/bin/python manage.py migrate &&
      /opt/netbox/venv/bin/python manage.py runserver 0.0.0.0:8080"
    environment:
      - DJANGO_SUPERUSER_PASSWORD=supersecretpassword
    ports:
      - 8000:8080
    volumes:
      - ./configuration-plugin:/etc/netbox/config:z,ro
      - ./reports:/opt/netbox/netbox/reports:rw
      - ./scripts:/opt/netbox/netbox/scripts:rw
      - ./plugins/my-first-plugin:/opt/netbox/netbox/my-first-plugin
      - ./plugins/netbox_cesnet_services_plugin:/opt/netbox/netbox/netbox_cesnet_services_plugin  
      

  netbox-worker:
    <<: *common
    container_name: netbox-worker
    volumes:
      - ./configuration:/etc/netbox/config:z,ro
      - ./reports:/opt/netbox/netbox/reports:rw
      - ./scripts:/opt/netbox/netbox/scripts:rw
      
  netbox-housekeeping:
    <<: *common
    container_name: netbox-housekeeping
    volumes:
      - ./configuration:/etc/netbox/config:z,ro
      - ./reports:/opt/netbox/netbox/reports:rw
      - ./scripts:/opt/netbox/netbox/scripts:rw
      

[WendyZhang-sh]

This issue has been fixed in netbox-community/netbox-chart#261