AVDump Failed to Install

[BadThor]

Before submitting your issue, please verify the following.

• You're using the latest Stable or Daily version of Shoko Server.
• You're using the latest and correct (Stable:Stable, Daily:Daily) Web UI for the version of Shoko you're using.
• You've searched this repository (top of the page) for the issue you are reporting.

Shoko Version

5.1.0.0

Shoko Version - Daily Commit

94c6bb7

Install Type

Linux - TrueNAS Scale

Description

I just installed ShokoServer using the community plugin. I have my AVDump key and AniDB set up correctly (I believe), but when I try to dump I get the error "AVDump failed to install!"

Checking the logs, I see this:

System.Net.Http.HttpRequestException: Name or service not known (files.shokoanime.com:443): System.Net.Http.HttpRequestException: Name or service not known (files.shokoanime.com:443) ---> System.Net.Sockets.SocketException (0xFFFDFFFF): Name or service not known at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpClient.GetStreamAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at Shoko.Server.Utilities.AVDumpHelper.PrepareAVDump(Boolean force) in /usr/src/app/source/Shoko.Server/Utilities/AVDumpHelper.cs:line 384

Not sure if I just set something up incorrectly or what. I am also unable ping/telnet the adress throught the error

telnet files.shokoanime.com 443
files.shokoanime.com: nodename nor servname provided, or not known

Steps to Reproduce

1. Install TrueNAS Scale Community Plugin for ShokoServer
2. Try to use the AVDump feature

Log File

System.Net.Http.HttpRequestException: Name or service not known (files.shokoanime.com:443): System.Net.Http.HttpRequestException: Name or service not known (files.shokoanime.com:443) ---> System.Net.Sockets.SocketException (0xFFFDFFFF): Name or service not known at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpClient.GetStreamAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at Shoko.Server.Utilities.AVDumpHelper.PrepareAVDump(Boolean force) in /usr/src/app/source/Shoko.Server/Utilities/AVDumpHelper.cs:line 384

Screenshots

No response

[da3dsoul]

We are aware. Our file host is currently down. You can extract this zip in a place that I don't immediately recall.

avdump3_8294_custom.zip

[da3dsoul]

I think it's something like ~/.shoko/Shoko.CLI/AVDump

[BadThor]

I placed it in the Shoko.CLI directory I found, and it doesn't seem to be working, but I will keep looking around. If the file host will be up soon-ish it isn't that big of a deal either

[Cazzar]

It should be the directory mentioned in the container.

for example:

root@848a1744dec1:/usr/src/app/build# ls ~shoko/.shoko/Shoko.CLI/AVDump/
AVD3AniDBModule.dll           MediaInfo-aarch64.so
AVD3AniDBModule.dll.app.hash  MediaInfo-x64.so
AVD3AniDBModule.dll.app.id    MediaInfo.dll
AVD3AniDBModule.dll.mil.hash  Microsoft.CodeAnalysis.CSharp.Scripting.dll
AVDump3CL                     Microsoft.CodeAnalysis.CSharp.dll
AVDump3CL.dll                 Microsoft.CodeAnalysis.Scripting.dll
AVDump3CL.exe                 Microsoft.CodeAnalysis.dll
AVDump3CL.runtimeconfig.json  Microsoft.Extensions.DependencyInjection.Abstractions.dll
AVDump3Lib.dll                Microsoft.Extensions.DependencyInjection.dll
AVDump3NativeLib-aarch64.so   Microsoft.Extensions.Logging.Abstractions.dll
AVDump3NativeLib-x64.so       Microsoft.Extensions.Logging.dll
AVDump3NativeLib.dll          Microsoft.Extensions.Options.dll
BXmlLib.dll                   Microsoft.Extensions.Primitives.dll
ExtKnot.StringInvariants.dll  Serilog.Extensions.Logging.dll
Ionic.Zlib.Core.dll           Serilog.dll

[BadThor]

Yeah, that is where I extracted it this morning

# ls -la ~shoko/.shoko/Shoko.CLI/AVDump
total 20403
drwxr-x--- 2 shoko shokogroup      32 Aug 15 06:01 .
drwxrwxr-x 8 shoko shokogroup      10 Aug 15 06:03 ..
-rw-r----- 1 shoko shokogroup   75776 Aug 15 06:00 AVD3AniDBModule.dll
-rw-r----- 1 shoko shokogroup      40 Aug 15 06:00 AVD3AniDBModule.dll.app.hash
-rw-r----- 1 shoko shokogroup       4 Aug 15 06:00 AVD3AniDBModule.dll.app.id
-rw-r----- 1 shoko shokogroup      40 Aug 15 06:00 AVD3AniDBModule.dll.mil.hash
-rw-r----- 1 shoko shokogroup  142840 Aug 15 06:00 AVDump3CL
-rw-r----- 1 shoko shokogroup  159744 Aug 15 06:00 AVDump3CL.dll
-rw-r----- 1 shoko shokogroup  260608 Aug 15 06:00 AVDump3CL.exe
-rw-r----- 1 shoko shokogroup     242 Aug 15 06:00 AVDump3CL.runtimeconfig.json
-rw-r----- 1 shoko shokogroup  433152 Aug 15 06:00 AVDump3Lib.dll
-rw-r----- 1 shoko shokogroup   48536 Aug 15 06:00 AVDump3NativeLib-aarch64.so
-rw-r----- 1 shoko shokogroup   60728 Aug 15 06:00 AVDump3NativeLib-x64.so
-rw-r----- 1 shoko shokogroup   47616 Aug 15 06:00 AVDump3NativeLib.dll
-rw-r----- 1 shoko shokogroup  117760 Aug 15 06:00 BXmlLib.dll
-rw-r----- 1 shoko shokogroup    9728 Aug 15 06:00 ExtKnot.StringInvariants.dll
-rw-r----- 1 shoko shokogroup   88064 Aug 15 06:00 Ionic.Zlib.Core.dll
-rw-r----- 1 shoko shokogroup 8899728 Aug 15 06:01 MediaInfo-aarch64.so
-rw-r----- 1 shoko shokogroup 9186800 Aug 15 06:01 MediaInfo-x64.so
-rw-r----- 1 shoko shokogroup 6333392 Aug 15 06:00 MediaInfo.dll
-rw-r----- 1 shoko shokogroup   32384 Aug 15 06:00 Microsoft.CodeAnalysis.CSharp.Scripting.dll
-rw-r----- 1 shoko shokogroup 5831808 Aug 15 06:01 Microsoft.CodeAnalysis.CSharp.dll
-rw-r----- 1 shoko shokogroup  136296 Aug 15 06:00 Microsoft.CodeAnalysis.Scripting.dll
-rw-r----- 1 shoko shokogroup 2645632 Aug 15 06:00 Microsoft.CodeAnalysis.dll
-rw-r----- 1 shoko shokogroup   43632 Aug 15 06:00 Microsoft.Extensions.DependencyInjection.Abstractions.dll
-rw-r----- 1 shoko shokogroup   81536 Aug 15 06:01 Microsoft.Extensions.DependencyInjection.dll
-rw-r----- 1 shoko shokogroup   46600 Aug 15 06:01 Microsoft.Extensions.Logging.Abstractions.dll
-rw-r----- 1 shoko shokogroup   30728 Aug 15 06:01 Microsoft.Extensions.Logging.dll
-rw-r----- 1 shoko shokogroup   26120 Aug 15 06:01 Microsoft.Extensions.Options.dll
-rw-r----- 1 shoko shokogroup   33288 Aug 15 06:01 Microsoft.Extensions.Primitives.dll
-rw-r----- 1 shoko shokogroup   17408 Aug 15 06:01 Serilog.Extensions.Logging.dll
-rw-r----- 1 shoko shokogroup  126464 Aug 15 06:01 Serilog.dll

But still getting

I did bring the container down and back up after extracting - and even tried to directly mount the extraced folder as a mount path to it, but still getting the install failed error

[da3dsoul]

Mark as executable maybe? The AVDump3CL ones specifically

[BadThor]

Okay, just tried that. Even completed deleted the Docker/volumes and started fresh.

AVDump is marked as executable

# ls -la ~shoko/.shoko/Shoko.CLI/AVDump
total 20403
drwxr-x--x+ 2 shoko shokogroup      32 Aug 15 08:43 .
drwxr-xr-x  8 shoko shokogroup      10 Aug 15 09:28 ..
-rwxr-x--x  1 shoko shokogroup   75776 Aug 15 08:43 AVD3AniDBModule.dll
-rwxr-x--x  1 shoko shokogroup      40 Aug 15 08:43 AVD3AniDBModule.dll.app.hash
-rwxr-x--x  1 shoko shokogroup       4 Aug 15 08:43 AVD3AniDBModule.dll.app.id
-rwxr-x--x  1 shoko shokogroup      40 Aug 15 08:43 AVD3AniDBModule.dll.mil.hash
-rwxr-x--x  1 shoko shokogroup  142840 Aug 15 08:42 AVDump3CL
-rwxr-x--x  1 shoko shokogroup  159744 Aug 15 08:42 AVDump3CL.dll
-rwxr-x--x  1 shoko shokogroup  260608 Aug 15 08:42 AVDump3CL.exe
-rwxr-x--x  1 shoko shokogroup     242 Aug 15 08:42 AVDump3CL.runtimeconfig.json
-rwxr-x--x  1 shoko shokogroup  433152 Aug 15 08:42 AVDump3Lib.dll
-rwxr-x--x  1 shoko shokogroup   48536 Aug 15 08:42 AVDump3NativeLib-aarch64.so
-rwxr-x--x  1 shoko shokogroup   60728 Aug 15 08:42 AVDump3NativeLib-x64.so
-rwxr-x--x  1 shoko shokogroup   47616 Aug 15 08:42 AVDump3NativeLib.dll
-rwxr-x--x  1 shoko shokogroup  117760 Aug 15 08:42 BXmlLib.dll
-rwxr-x--x  1 shoko shokogroup    9728 Aug 15 08:42 ExtKnot.StringInvariants.dll
-rwxr-x--x  1 shoko shokogroup   88064 Aug 15 08:42 Ionic.Zlib.Core.dll
-rwxr-x--x  1 shoko shokogroup 8899728 Aug 15 08:43 MediaInfo-aarch64.so
-rwxr-x--x  1 shoko shokogroup 9186800 Aug 15 08:43 MediaInfo-x64.so
-rwxr-x--x  1 shoko shokogroup 6333392 Aug 15 08:43 MediaInfo.dll
-rwxr-x--x  1 shoko shokogroup   32384 Aug 15 08:42 Microsoft.CodeAnalysis.CSharp.Scripting.dll
-rwxr-x--x  1 shoko shokogroup 5831808 Aug 15 08:43 Microsoft.CodeAnalysis.CSharp.dll
-rwxr-x--x  1 shoko shokogroup  136296 Aug 15 08:43 Microsoft.CodeAnalysis.Scripting.dll
-rwxr-x--x  1 shoko shokogroup 2645632 Aug 15 08:43 Microsoft.CodeAnalysis.dll
-rwxr-x--x  1 shoko shokogroup   43632 Aug 15 08:43 Microsoft.Extensions.DependencyInjection.Abstractions.dll
-rwxr-x--x  1 shoko shokogroup   81536 Aug 15 08:43 Microsoft.Extensions.DependencyInjection.dll
-rwxr-x--x  1 shoko shokogroup   46600 Aug 15 08:43 Microsoft.Extensions.Logging.Abstractions.dll
-rwxr-x--x  1 shoko shokogroup   30728 Aug 15 08:43 Microsoft.Extensions.Logging.dll
-rwxr-x--x  1 shoko shokogroup   26120 Aug 15 08:43 Microsoft.Extensions.Options.dll
-rwxr-x--x  1 shoko shokogroup   33288 Aug 15 08:43 Microsoft.Extensions.Primitives.dll
-rwxr-x--x  1 shoko shokogroup   17408 Aug 15 08:43 Serilog.Extensions.Logging.dll
-rwxr-x--x  1 shoko shokogroup  126464 Aug 15 08:43 Serilog.dll

Still getting the Failed to Install

[t0815]

Yes, same issue here. For some reason, shoko trys to DL Avdump again, even if it's located in the ./AVDump -Folder, which fails due to mentioned fileserver outage. I checked logs and it seems, if avdump fails to dump a file (i.e (in this case) fails to connect to anidb),

Failed to complete AVDump session 2: Files: I:\Clevatess Majuu no Ou to Akago to Shikabane no Yuusha\Season 01\[Ironclad] Clevatess - S01E07 [1080p.AV1] Clevatess (Weekly).mkv Standard Output: Testing anidb connection Format Error: Could not parse Id: Check for program updates. DumpError: PacketFormat / Could not parse Id Pending AniDB reply timed out Format Error: Could not parse Id: Check for program updates. DumpError: PacketFormat / Could not parse Id Pending AniDB reply timed out Format Error: Could not parse Id: Check for program updates. DumpError: PacketFormat / Could not parse Id Pending AniDB reply timed out Format Error: Could not parse Id: Check for program updates. DumpError: PacketFormat / Could not parse Id Pending AniDB reply timed out Startup Cancel: Couldn't establish session with anidb (Timeout) 

shoko assumes something is wrong with AVDump installation and trys to re-download it from

files.shokoanime.com:443

which fails, creating an endless loop :)

[cchinchole]

I got it to work by taking the downloaded .zip and placing it in Shoko.CLI as "avdump.zip". It seems that it wants to extract the archive itself at initial launch.

[t0815]

Yes, that will work, but sometimes avdump can't connect or is getting a timeout, if that happens, shoko is thinking avdump needs an update, and tries to download avdump again - which of course fails. I assume, it's because of the message in the returned error:

Could not parse Id Pending AniDB reply timed out Format Error: Could not parse Id: Check for program updates.

Not sure how to handle this... but i dont think shoko should assume automatically, that avdump needs to update.

[cchinchole]

Yes, that will work, but sometimes avdump can't connect or is getting a timeout, if that happens, shoko is thinking avdump needs an update, and tries to download avdump again - which of course fails. I assume, it's because of the message in the returned error:

Could not parse Id Pending AniDB reply timed out Format Error: Could not parse Id: Check for program updates.

Not sure how to handle this... but i dont think shoko should assume automatically, that avdump needs to update.

The times I've had this, I restarted the server and tried again. Or if this wouldn't work, I'd wait awhile. The issue seems to occur randomly, but I've been able to dump anything necessary. IIRC this timeout happens within the anidb library they're using. Doesn't seem like it's their issue to fix, rather how it's handled perhaps; as you mentioned I don't believe it should be assumed avdump requires an update unless there's a version mismatch or the executable is corrupt.

[revam]

Yes, that will work, but sometimes avdump can't connect or is getting a timeout, if that happens, shoko is thinking avdump needs an update, and tries to download avdump again - which of course fails. I assume, it's because of the message in the returned error:

Could not parse Id Pending AniDB reply timed out Format Error: Could not parse Id: Check for program updates.

Not sure how to handle this... but i dont think shoko should assume automatically, that avdump needs to update.

There is no code to update avd3 because it receives a timeout during execution or otherwise fails to run the already installed version.

The only times it will try to upgrade are if you have a lower version installed compared to what is expected or if it was not installed in the first place before a dumping session is started.

[t0815]

hmm.. how can this happen than? I did test this, and downloaded the latest version from avdump -wiki page on anidb, placed the zip into shokoserver folder under programdata. Shoko is extracting the zip... avdump fails.... shoko deletes all contents in AVDump folder and tries to download agian, which fails too.

So, i assume the problem might be, because shoko cant reach the file-server, it thinks current version is outdated, deletes folder, tries to dl again, fails.....