Mac OS: tello_opencv.go error writing to ffmpeg stdin

[emf42]

Hey, I've been trying to get the tello_opencv.go example working.

The issue is the ffmpeg.StdinPipe() doesn't seem to be a valid pipe.

When the code tries to do ffmpegIn.Write(pkt), it logs the error write |1: broken pipe, causing io.ReadFull(ffmpegOut, buf) to reach EOF.

I'm running MacOS 10.13.4.

The problem might be with the ffmpeg command not properly recognizing that input will be coming from pipe 0 (stdin).
Do different OS need different args formatting for ffmpeg input/output?

[deadprogram]

Hi @emf42 that is a very interesting question. According to the docs for ffmpeg that is the default pipe in, however I think for sure worth exploring.

[emf42]

@deadprogram, ya I've been trying all sorts of args formatting but still get the same error. Did you have to do anything special for yours to work?

[deadprogram]

Just the args in the code itself here:
https://github.com/hybridgroup/gobot/blob/master/examples/tello_opencv.go#L37-L38

One thing perhaps worth trying would be running that command just on your command line and seeing what happens, such as some error.

ffmpeg -i pipe:0 -pix_fmt bgr24 -vcodec rawvideo -an -sn -s 960x720 -f rawvideo pipe:1

[emf42]

$ ffmpeg -i pipe:0 -pix_fmt bgr24 -vcodec rawvideo -an -sn -s 960x720 -f rawvideo pipe:1
ffmpeg version 4.0 Copyright (c) 2000-2018 the FFmpeg developers
  built with Apple LLVM version 9.1.0 (clang-902.0.39.1)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.0 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-gpl --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma
  libavutil      56. 14.100 / 56. 14.100
  libavcodec     58. 18.100 / 58. 18.100
  libavformat    58. 12.100 / 58. 12.100
  libavdevice    58.  3.100 / 58.  3.100
  libavfilter     7. 16.100 /  7. 16.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  1.100 /  5.  1.100
  libswresample   3.  1.100 /  3.  1.100
  libpostproc    55.  1.100 / 55.  1.100

Theres the output. Execution does not halt.

[deadprogram]

How about this: check to see if you are even getting video data back from the Tello, by adding a fmt.Println("here") right before this line: https://github.com/hybridgroup/gobot/blob/master/examples/tello_opencv.go#L75

Perhaps you are not even getting video frame data from drone.

[emf42]

I am getting the video data.
The pipe closed error stems from the ffmpegIn.Write(pkt) in that same block.

[emf42]

@deadprogram I just tried it a few more times, and now I am getting this error:

tello_opencv[2836:86136] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'nextEventMatchingMask should only be called from the Main Thread!'

[deadprogram]

No idea on that, sorry. Obviously some internal macOS error, but ?

[emf42]

It looks to be an openCV issue, not macOS

tello_opencv[3236:110730] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'nextEventMatchingMask should only be called from the Main Thread!'
*** First throw call stack:
(
	0   CoreFoundation                      0x00007fff401af32b __exceptionPreprocess + 171
	1   libobjc.A.dylib                     0x00007fff67829c76 objc_exception_throw + 48
	2   AppKit                              0x00007fff3de5b297 -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 4167
	3   libopencv_highgui.3.4.dylib         0x0000000005016a2c cvWaitKey + 351
	4   libopencv_highgui.3.4.dylib         0x0000000005012149 _ZN2cv9waitKeyExEi + 34
	5   libopencv_highgui.3.4.dylib         0x00000000050121af _ZN2cv7waitKeyEi + 34
	6   tello_opencv                        0x00000000041194d3 Window_WaitKey + 19
	7   tello_opencv                        0x000000000410915c _cgo_4c3040cb4e6c_Cfunc_Window_WaitKey + 28
	8   tello_opencv                        0x0000000004054f90 runtime.asmcgocall + 112
)
libc++abi.dylib: terminating with uncaught exception of type NSException
SIGABRT: abort
PC=0x7fff68593b6e m=12 sigcode=0

[deadprogram]

Generally the main thread and highgui windows are handled by https://github.com/hybridgroup/gocv/blob/master/highgui.go#L34

Since that problem just started, you might want to see if some lingering process is running?

[emf42]

Heres something interesting. If I remove the go func and take everything out of the work function, there are no problems writing the video data to ffmpegIn.
https://pastebin.com/XTD2WE1R

When I try to add back the go function for reading from ffmpegout, I get the same error as above, but theres no pipe errors.

[deadprogram]

Do you get the message "Connected" in your log?

[emf42]

Yes I do.

[emf42]

Have you got any other ideas on how to create an OpenCV image mat from the video data? I havent been able to find anything else to work.