Input/Output Nodes for Freezing the Model

[angela-zhao]

I'm trying to freeze the Quick-thoughts model but am having trouble identifying the input and output nodes. I believe the output node should be "word_embeddings" but am not 100% sure.

My end goal is to be able to use the encoder from a frozen version of the model.

[angela-zhao]

I used the summarize_graph tool provided by tensorflow and the output says that no inputs were spotted. Here is the full output:

No inputs spotted.
Found 57 variables: (name=word_embedding, type=float(1), shape=[50001,620]) (name=word_embedding_out, type=float(1), shape=[50001,620]) (name=encoder/fw/gru_cell/gates/kernel, type=float(1), shape=[1220,1200]) (name=encoder/fw/gru_cell/gates/bias, type=float(1), shape=[1200]) (name=encoder/fw/gru_cell/candidate/kernel, type=float(1), shape=[1220,600]) (name=encoder/fw/gru_cell/candidate/bias, type=float(1), shape=[600]) (name=encoder/bw/gru_cell/gates/kernel, type=float(1), shape=[1220,1200]) (name=encoder/bw/gru_cell/gates/bias, type=float(1), shape=[1200]) (name=encoder/bw/gru_cell/candidate/kernel, type=float(1), shape=[1220,600]) (name=encoder/bw/gru_cell/candidate/bias, type=float(1), shape=[600]) (name=encoder_out/fw/gru_cell/gates/kernel, type=float(1), shape=[1220,1200]) (name=encoder_out/fw/gru_cell/gates/bias, type=float(1), shape=[1200]) (name=encoder_out/fw/gru_cell/candidate/kernel, type=float(1), shape=[1220,600]) (name=encoder_out/fw/gru_cell/candidate/bias, type=float(1), shape=[600]) (name=encoder_out/bw/gru_cell/gates/kernel, type=float(1), shape=[1220,1200]) (name=encoder_out/bw/gru_cell/gates/bias, type=float(1), shape=[1200]) (name=encoder_out/bw/gru_cell/candidate/kernel, type=float(1), shape=[1220,600]) (name=encoder_out/bw/gru_cell/candidate/bias, type=float(1), shape=[600]) (name=global_step, type=int64(9), shape=[]) (name=beta1_power, type=float(1), shape=[]) (name=beta2_power, type=float(1), shape=[]) (name=word_embedding/Adam, type=float(1), shape=[50001,620]) (name=word_embedding/Adam_1, type=float(1), shape=[50001,620]) (name=word_embedding_out/Adam, type=float(1), shape=[50001,620]) (name=word_embedding_out/Adam_1, type=float(1), shape=[50001,620]) (name=encoder/fw/gru_cell/gates/kernel/Adam, type=float(1), shape=[1220,1200]) (name=encoder/fw/gru_cell/gates/kernel/Adam_1, type=float(1), shape=[1220,1200]) (name=encoder/fw/gru_cell/gates/bias/Adam, type=float(1), shape=[1200]) (name=encoder/fw/gru_cell/gates/bias/Adam_1, type=float(1), shape=[1200]) (name=encoder/fw/gru_cell/candidate/kernel/Adam, type=float(1), shape=[1220,600]) (name=encoder/fw/gru_cell/candidate/kernel/Adam_1, type=float(1), shape=[1220,600]) (name=encoder/fw/gru_cell/candidate/bias/Adam, type=float(1), shape=[600]) (name=encoder/fw/gru_cell/candidate/bias/Adam_1, type=float(1), shape=[600]) (name=encoder/bw/gru_cell/gates/kernel/Adam, type=float(1), shape=[1220,1200]) (name=encoder/bw/gru_cell/gates/kernel/Adam_1, type=float(1), shape=[1220,1200]) (name=encoder/bw/gru_cell/gates/bias/Adam, type=float(1), shape=[1200]) (name=encoder/bw/gru_cell/gates/bias/Adam_1, type=float(1), shape=[1200]) (name=encoder/bw/gru_cell/candidate/kernel/Adam, type=float(1), shape=[1220,600]) (name=encoder/bw/gru_cell/candidate/kernel/Adam_1, type=float(1), shape=[1220,600]) (name=encoder/bw/gru_cell/candidate/bias/Adam, type=float(1), shape=[600]) (name=encoder/bw/gru_cell/candidate/bias/Adam_1, type=float(1), shape=[600]) (name=encoder_out/fw/gru_cell/gates/kernel/Adam, type=float(1), shape=[1220,1200]) (name=encoder_out/fw/gru_cell/gates/kernel/Adam_1, type=float(1), shape=[1220,1200]) (name=encoder_out/fw/gru_cell/gates/bias/Adam, type=float(1), shape=[1200]) (name=encoder_out/fw/gru_cell/gates/bias/Adam_1, type=float(1), shape=[1200]) (name=encoder_out/fw/gru_cell/candidate/kernel/Adam, type=float(1), shape=[1220,600]) (name=encoder_out/fw/gru_cell/candidate/kernel/Adam_1, type=float(1), shape=[1220,600]) (name=encoder_out/fw/gru_cell/candidate/bias/Adam, type=float(1), shape=[600]) (name=encoder_out/fw/gru_cell/candidate/bias/Adam_1, type=float(1), shape=[600]) (name=encoder_out/bw/gru_cell/gates/kernel/Adam, type=float(1), shape=[1220,1200]) (name=encoder_out/bw/gru_cell/gates/kernel/Adam_1, type=float(1), shape=[1220,1200]) (name=encoder_out/bw/gru_cell/gates/bias/Adam, type=float(1), shape=[1200]) (name=encoder_out/bw/gru_cell/gates/bias/Adam_1, type=float(1), shape=[1200]) (name=encoder_out/bw/gru_cell/candidate/kernel/Adam, type=float(1), shape=[1220,600]) (name=encoder_out/bw/gru_cell/candidate/kernel/Adam_1, type=float(1), shape=[1220,600]) (name=encoder_out/bw/gru_cell/candidate/bias/Adam, type=float(1), shape=[600]) (name=encoder_out/bw/gru_cell/candidate/bias/Adam_1, type=float(1), shape=[600])
Found 99 possible outputs: (name=filename_queue/filename_queue_EnqueueMany, op=QueueEnqueueManyV2) (name=filename_queue/filename_queue_Close, op=QueueCloseV2) (name=filename_queue/filename_queue_Close_1, op=QueueCloseV2) (name=fifo_input_queue_enqueue, op=QueueEnqueueV2) (name=fifo_input_queue_Close, op=QueueCloseV2) (name=fifo_input_queue_Close_1, op=QueueCloseV2) (name=encoder/encoder/fw/fw/GRUCellZeroState/ExpandDims_1, op=ExpandDims) (name=encoder/encoder/fw/fw/Min, op=Min) (name=encoder/encoder/fw/fw/while/Exit, op=Exit) (name=encoder/encoder/fw/fw/while/Exit_1, op=Exit) (name=encoder/encoder/fw/fw/transpose_1, op=Transpose) (name=encoder/encoder/bw/bw/GRUCellZeroState/ExpandDims_1, op=ExpandDims) (name=encoder/encoder/bw/bw/Min, op=Min) (name=encoder/encoder/bw/bw/while/Exit, op=Exit) (name=encoder/encoder/bw/bw/while/Exit_1, op=Exit) (name=encoder/ReverseSequence, op=ReverseSequence) (name=encoder_out/encoder_out/fw/fw/GRUCellZeroState/ExpandDims_1, op=ExpandDims) (name=encoder_out/encoder_out/fw/fw/Min, op=Min) (name=encoder_out/encoder_out/fw/fw/while/Exit, op=Exit) (name=encoder_out/encoder_out/fw/fw/while/Exit_1, op=Exit) (name=encoder_out/encoder_out/fw/fw/transpose_1, op=Transpose) (name=encoder_out/encoder_out/bw/bw/GRUCellZeroState/ExpandDims_1, op=ExpandDims) (name=encoder_out/encoder_out/bw/bw/Min, op=Min) (name=encoder_out/encoder_out/bw/bw/while/Exit, op=Exit) (name=encoder_out/encoder_out/bw/bw/while/Exit_1, op=Exit) (name=encoder_out/ReverseSequence, op=ReverseSequence) (name=strided_slice, op=StridedSlice) (name=strided_slice_1, op=StridedSlice) (name=gradients/b_count_3, op=Exit) (name=gradients/b_count_7, op=Exit) (name=gradients/b_count_11, op=Exit) (name=gradients/b_count_15, op=Exit) (name=gradients/zeros_like, op=ZerosLike) (name=gradients/softmax_cross_entropy_with_logits_sg_grad/tuple/control_dependency_1, op=Identity) (name=gradients/MatrixSetDiag_grad/tuple/control_dependency_1, op=Identity) (name=gradients/encoder/concat_grad/Shape, op=Shape) (name=gradients/encoder_out/concat_grad/Shape, op=Shape) (name=gradients/encoder/encoder/fw/fw/while/Exit_2_grad/b_exit, op=Enter) (name=gradients/encoder/encoder/bw/bw/while/Exit_2_grad/b_exit, op=Enter) (name=gradients/encoder_out/encoder_out/fw/fw/while/Exit_2_grad/b_exit, op=Enter) (name=gradients/encoder_out/encoder_out/bw/bw/while/Exit_2_grad/b_exit, op=Enter) (name=gradients/encoder/encoder/fw/fw/while/Enter_3_grad/Exit, op=Exit) (name=gradients/encoder/encoder/bw/bw/while/Enter_3_grad/Exit, op=Exit) (name=gradients/encoder_out/encoder_out/fw/fw/while/Enter_3_grad/Exit, op=Exit) (name=gradients/encoder_out/encoder_out/bw/bw/while/Enter_3_grad/Exit, op=Exit) (name=gradients/encoder/encoder/fw/fw/while/gru_cell/sub_grad/tuple/control_dependency, op=Identity) (name=gradients/encoder/encoder/bw/bw/while/gru_cell/sub_grad/tuple/control_dependency, op=Identity) (name=gradients/encoder_out/encoder_out/fw/fw/while/gru_cell/sub_grad/tuple/control_dependency, op=Identity) (name=gradients/encoder_out/encoder_out/bw/bw/while/gru_cell/sub_grad/tuple/control_dependency, op=Identity) (name=gradients/encoder/encoder/fw/fw/while/gru_cell/concat_1_grad/Shape, op=Shape) (name=gradients/encoder/encoder/bw/bw/while/gru_cell/concat_1_grad/Shape, op=Shape) (name=gradients/encoder_out/encoder_out/fw/fw/while/gru_cell/concat_1_grad/Shape, op=Shape) (name=gradients/encoder_out/encoder_out/bw/bw/while/gru_cell/concat_1_grad/Shape, op=Shape) (name=gradients/encoder/encoder/fw/fw/while/gru_cell/concat_grad/Shape, op=Shape) (name=gradients/encoder/encoder/bw/bw/while/gru_cell/concat_grad/Shape, op=Shape) (name=gradients/encoder_out/encoder_out/fw/fw/while/gru_cell/concat_grad/Shape, op=Shape) (name=gradients/encoder_out/encoder_out/bw/bw/while/gru_cell/concat_grad/Shape, op=Shape) (name=gradients/encoder/encoder/fw/fw/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/tuple/control_dependency_1, op=Identity) (name=gradients/encoder/encoder/bw/bw/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/tuple/control_dependency_1, op=Identity) (name=gradients/encoder_out/encoder_out/fw/fw/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/tuple/control_dependency_1, op=Identity) (name=gradients/encoder_out/encoder_out/bw/bw/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/tuple/control_dependency_1, op=Identity) (name=encoder/fw/gru_cell/gates/kernel/Adam/read, op=Identity) (name=encoder/fw/gru_cell/gates/kernel/Adam_1/read, op=Identity) (name=encoder/fw/gru_cell/gates/bias/Adam/read, op=Identity) (name=encoder/fw/gru_cell/gates/bias/Adam_1/read, op=Identity) (name=encoder/fw/gru_cell/candidate/kernel/Adam/read, op=Identity) (name=encoder/fw/gru_cell/candidate/kernel/Adam_1/read, op=Identity) (name=encoder/fw/gru_cell/candidate/bias/Adam/read, op=Identity) (name=encoder/fw/gru_cell/candidate/bias/Adam_1/read, op=Identity) (name=encoder/bw/gru_cell/gates/kernel/Adam/read, op=Identity) (name=encoder/bw/gru_cell/gates/kernel/Adam_1/read, op=Identity) (name=encoder/bw/gru_cell/gates/bias/Adam/read, op=Identity) (name=encoder/bw/gru_cell/gates/bias/Adam_1/read, op=Identity) (name=encoder/bw/gru_cell/candidate/kernel/Adam/read, op=Identity) (name=encoder/bw/gru_cell/candidate/kernel/Adam_1/read, op=Identity) (name=encoder/bw/gru_cell/candidate/bias/Adam/read, op=Identity) (name=encoder/bw/gru_cell/candidate/bias/Adam_1/read, op=Identity) (name=encoder_out/fw/gru_cell/gates/kernel/Adam/read, op=Identity) (name=encoder_out/fw/gru_cell/gates/kernel/Adam_1/read, op=Identity) (name=encoder_out/fw/gru_cell/gates/bias/Adam/read, op=Identity) (name=encoder_out/fw/gru_cell/gates/bias/Adam_1/read, op=Identity) (name=encoder_out/fw/gru_cell/candidate/kernel/Adam/read, op=Identity) (name=encoder_out/fw/gru_cell/candidate/kernel/Adam_1/read, op=Identity) (name=encoder_out/fw/gru_cell/candidate/bias/Adam/read, op=Identity) (name=encoder_out/fw/gru_cell/candidate/bias/Adam_1/read, op=Identity) (name=encoder_out/bw/gru_cell/gates/kernel/Adam/read, op=Identity) (name=encoder_out/bw/gru_cell/gates/kernel/Adam_1/read, op=Identity) (name=encoder_out/bw/gru_cell/gates/bias/Adam/read, op=Identity) (name=encoder_out/bw/gru_cell/gates/bias/Adam_1/read, op=Identity) (name=encoder_out/bw/gru_cell/candidate/kernel/Adam/read, op=Identity) (name=encoder_out/bw/gru_cell/candidate/kernel/Adam_1/read, op=Identity) (name=encoder_out/bw/gru_cell/candidate/bias/Adam/read, op=Identity) (name=encoder_out/bw/gru_cell/candidate/bias/Adam_1/read, op=Identity) (name=train_op/control_dependency, op=Identity) (name=save/control_dependency, op=Identity) (name=init_ops/report_uninitialized_variables/boolean_mask/GatherV2, op=GatherV2) (name=Merge/MergeSummary, op=MergeSummary) (name=train_step/GreaterEqual, op=GreaterEqual) (name=train_step/Equal, op=Equal)
Found 8964531 (8.96M) const parameters, 0 (0) variable parameters, and 838 control_edges
33 nodes assigned to device '/device:CPU:0'Op types used: 640 Const, 284 Enter, 234 Identity, 120 Assign, 102 Mul, 84 StackPopV2, 84 StackPushV2, 84 StackV2, 75 Shape, 69 NoOp, 60 Sum, 57 IsVariableInitialized, 57 VariableV2, 52 Reshape, 48 Add, 48 Switch, 48 NextIteration, 48 Exit, 48 Merge, 39 Fill, 36 ConcatV2, 31 Sub, 27 MatMul, 25 StridedSlice, 25 Slice, 23 RealDiv, 23 Maximum, 22 Sqrt, 20 BroadcastGradientArgs, 16 ApplyAdam, 16 ExpandDims, 16 Select, 16 Range, 14 AddN, 12 Pack, 12 Transpose, 11 FloorMod, 10 ConcatOffset, 10 ShapeN, 10 RandomUniform, 9 GreaterEqual, 8 TensorArrayGradV3, 8 BiasAdd, 8 TensorArrayGatherV3, 8 BiasAddGrad, 8 LoopCond, 8 TensorArrayWriteV3, 8 Less, 8 TensorArrayV3, 6 ReverseSequence, 5 Neg, 5 Cast, 5 ZerosLike, 5 Assert, 5 Min, 5 Equal, 4 InvertPermutation, 4 ControlTrigger, 4 Minimum, 4 All, 4 QueueCloseV2, 4 Split, 4 TensorArraySizeV3, 4 TensorArrayScatterV3, 4 Max, 4 TensorArrayReadV3, 4 Tanh, 4 LogicalAnd, 4 SigmoidGrad, 4 ScatterAdd, 4 TanhGrad, 4 Sigmoid, 3 Prod, 3 GatherV2, 3 ScalarSummary, 2 MatrixSetDiag, 2 SparseToDense, 2 AssignSub, 2 Size, 2 Unique, 2 UnsortedSegmentSum, 2 FIFOQueueV2, 2 QueueSizeV2, 1 StopGradient, 1 FloorDiv, 1 Greater, 1 Squeeze, 1 ParseExample, 1 TFRecordReaderV2, 1 SoftmaxCrossEntropyWithLogits, 1 LogicalNot, 1 CheckNumerics, 1 MatrixDiagPart, 1 SaveV2, 1 RestoreV2, 1 Mean, 1 MergeSummary, 1 AssignAdd, 1 Tile, 1 ReaderReadV2, 1 QueueEnqueueV2, 1 QueueEnqueueManyV2, 1 QueueDequeueManyV2, 1 Where, 1 LogSoftmax
To use with tensorflow/tools/benchmark:benchmark_model try these arguments:
bazel run tensorflow/tools/benchmark:benchmark_model -- --graph=/Users/angela.zhao/Desktop/text-similarity-deploy/results/BS400-W620-S1200-case-bidir/train/graph.pbtxt --show_flops --input_layer=word_embedding,word_embedding_out,encoder/fw/gru_cell/gates/kernel,encoder/fw/gru_cell/gates/bias,encoder/fw/gru_cell/candidate/kernel,encoder/fw/gru_cell/candidate/bias,encoder/bw/gru_cell/gates/kernel,encoder/bw/gru_cell/gates/bias,encoder/bw/gru_cell/candidate/kernel,encoder/bw/gru_cell/candidate/bias,encoder_out/fw/gru_cell/gates/kernel,encoder_out/fw/gru_cell/gates/bias,encoder_out/fw/gru_cell/candidate/kernel,encoder_out/fw/gru_cell/candidate/bias,encoder_out/bw/gru_cell/gates/kernel,encoder_out/bw/gru_cell/gates/bias,encoder_out/bw/gru_cell/candidate/kernel,encoder_out/bw/gru_cell/candidate/bias,global_step,beta1_power,beta2_power,word_embedding/Adam,word_embedding/Adam_1,word_embedding_out/Adam,word_embedding_out/Adam_1,encoder/fw/gru_cell/gates/kernel/Adam,encoder/fw/gru_cell/gates/kernel/Adam_1,encoder/fw/gru_cell/gates/bias/Adam,encoder/fw/gru_cell/gates/bias/Adam_1,encoder/fw/gru_cell/candidate/kernel/Adam,encoder/fw/gru_cell/candidate/kernel/Adam_1,encoder/fw/gru_cell/candidate/bias/Adam,encoder/fw/gru_cell/candidate/bias/Adam_1,encoder/bw/gru_cell/gates/kernel/Adam,encoder/bw/gru_cell/gates/kernel/Adam_1,encoder/bw/gru_cell/gates/bias/Adam,encoder/bw/gru_cell/gates/bias/Adam_1,encoder/bw/gru_cell/candidate/kernel/Adam,encoder/bw/gru_cell/candidate/kernel/Adam_1,encoder/bw/gru_cell/candidate/bias/Adam,encoder/bw/gru_cell/candidate/bias/Adam_1,encoder_out/fw/gru_cell/gates/kernel/Adam,encoder_out/fw/gru_cell/gates/kernel/Adam_1,encoder_out/fw/gru_cell/gates/bias/Adam,encoder_out/fw/gru_cell/gates/bias/Adam_1,encoder_out/fw/gru_cell/candidate/kernel/Adam,encoder_out/fw/gru_cell/candidate/kernel/Adam_1,encoder_out/fw/gru_cell/candidate/bias/Adam,encoder_out/fw/gru_cell/candidate/bias/Adam_1,encoder_out/bw/gru_cell/gates/kernel/Adam,encoder_out/bw/gru_cell/gates/kernel/Adam_1,encoder_out/bw/gru_cell/gates/bias/Adam,encoder_out/bw/gru_cell/gates/bias/Adam_1,encoder_out/bw/gru_cell/candidate/kernel/Adam,encoder_out/bw/gru_cell/candidate/kernel/Adam_1,encoder_out/bw/gru_cell/candidate/bias/Adam,encoder_out/bw/gru_cell/candidate/bias/Adam_1 --input_layer_type=float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,int64,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float,float --input_layer_shape=50001,620:50001,620:1220,1200:1200:1220,600:600:1220,1200:1200:1220,600:600:1220,1200:1200:1220,600:600:1220,1200:1200:1220,600:600::::50001,620:50001,620:50001,620:50001,620:1220,1200:1220,1200:1200:1200:1220,600:1220,600:600:600:1220,1200:1220,1200:1200:1200:1220,600:1220,600:600:600:1220,1200:1220,1200:1200:1200:1220,600:1220,600:600:600:1220,1200:1220,1200:1200:1200:1220,600:1220,600:600:600 --output_layer=filename_queue/filename_queue_EnqueueMany,filename_queue/filename_queue_Close,filename_queue/filename_queue_Close_1,fifo_input_queue_enqueue,fifo_input_queue_Close,fifo_input_queue_Close_1,encoder/encoder/fw/fw/GRUCellZeroState/ExpandDims_1,encoder/encoder/fw/fw/Min,encoder/encoder/fw/fw/while/Exit,encoder/encoder/fw/fw/while/Exit_1,encoder/encoder/fw/fw/transpose_1,encoder/encoder/bw/bw/GRUCellZeroState/ExpandDims_1,encoder/encoder/bw/bw/Min,encoder/encoder/bw/bw/while/Exit,encoder/encoder/bw/bw/while/Exit_1,encoder/ReverseSequence,encoder_out/encoder_out/fw/fw/GRUCellZeroState/ExpandDims_1,encoder_out/encoder_out/fw/fw/Min,encoder_out/encoder_out/fw/fw/while/Exit,encoder_out/encoder_out/fw/fw/while/Exit_1,encoder_out/encoder_out/fw/fw/transpose_1,encoder_out/encoder_out/bw/bw/GRUCellZeroState/ExpandDims_1,encoder_out/encoder_out/bw/bw/Min,encoder_out/encoder_out/bw/bw/while/Exit,encoder_out/encoder_out/bw/bw/while/Exit_1,encoder_out/ReverseSequence,strided_slice,strided_slice_1,gradients/b_count_3,gradients/b_count_7,gradients/b_count_11,gradients/b_count_15,gradients/zeros_like,gradients/softmax_cross_entropy_with_logits_sg_grad/tuple/control_dependency_1,gradients/MatrixSetDiag_grad/tuple/control_dependency_1,gradients/encoder/concat_grad/Shape,gradients/encoder_out/concat_grad/Shape,gradients/encoder/encoder/fw/fw/while/Exit_2_grad/b_exit,gradients/encoder/encoder/bw/bw/while/Exit_2_grad/b_exit,gradients/encoder_out/encoder_out/fw/fw/while/Exit_2_grad/b_exit,gradients/encoder_out/encoder_out/bw/bw/while/Exit_2_grad/b_exit,gradients/encoder/encoder/fw/fw/while/Enter_3_grad/Exit,gradients/encoder/encoder/bw/bw/while/Enter_3_grad/Exit,gradients/encoder_out/encoder_out/fw/fw/while/Enter_3_grad/Exit,gradients/encoder_out/encoder_out/bw/bw/while/Enter_3_grad/Exit,gradients/encoder/encoder/fw/fw/while/gru_cell/sub_grad/tuple/control_dependency,gradients/encoder/encoder/bw/bw/while/gru_cell/sub_grad/tuple/control_dependency,gradients/encoder_out/encoder_out/fw/fw/while/gru_cell/sub_grad/tuple/control_dependency,gradients/encoder_out/encoder_out/bw/bw/while/gru_cell/sub_grad/tuple/control_dependency,gradients/encoder/encoder/fw/fw/while/gru_cell/concat_1_grad/Shape,gradients/encoder/encoder/bw/bw/while/gru_cell/concat_1_grad/Shape,gradients/encoder_out/encoder_out/fw/fw/while/gru_cell/concat_1_grad/Shape,gradients/encoder_out/encoder_out/bw/bw/while/gru_cell/concat_1_grad/Shape,gradients/encoder/encoder/fw/fw/while/gru_cell/concat_grad/Shape,gradients/encoder/encoder/bw/bw/while/gru_cell/concat_grad/Shape,gradients/encoder_out/encoder_out/fw/fw/while/gru_cell/concat_grad/Shape,gradients/encoder_out/encoder_out/bw/bw/while/gru_cell/concat_grad/Shape,gradients/encoder/encoder/fw/fw/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/tuple/control_dependency_1,gradients/encoder/encoder/bw/bw/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/tuple/control_dependency_1,gradients/encoder_out/encoder_out/fw/fw/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/tuple/control_dependency_1,gradients/encoder_out/encoder_out/bw/bw/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/tuple/control_dependency_1,encoder/fw/gru_cell/gates/kernel/Adam/read,encoder/fw/gru_cell/gates/kernel/Adam_1/read,encoder/fw/gru_cell/gates/bias/Adam/read,encoder/fw/gru_cell/gates/bias/Adam_1/read,encoder/fw/gru_cell/candidate/kernel/Adam/read,encoder/fw/gru_cell/candidate/kernel/Adam_1/read,encoder/fw/gru_cell/candidate/bias/Adam/read,encoder/fw/gru_cell/candidate/bias/Adam_1/read,encoder/bw/gru_cell/gates/kernel/Adam/read,encoder/bw/gru_cell/gates/kernel/Adam_1/read,encoder/bw/gru_cell/gates/bias/Adam/read,encoder/bw/gru_cell/gates/bias/Adam_1/read,encoder/bw/gru_cell/candidate/kernel/Adam/read,encoder/bw/gru_cell/candidate/kernel/Adam_1/read,encoder/bw/gru_cell/candidate/bias/Adam/read,encoder/bw/gru_cell/candidate/bias/Adam_1/read,encoder_out/fw/gru_cell/gates/kernel/Adam/read,encoder_out/fw/gru_cell/gates/kernel/Adam_1/read,encoder_out/fw/gru_cell/gates/bias/Adam/read,encoder_out/fw/gru_cell/gates/bias/Adam_1/read,encoder_out/fw/gru_cell/candidate/kernel/Adam/read,encoder_out/fw/gru_cell/candidate/kernel/Adam_1/read,encoder_out/fw/gru_cell/candidate/bias/Adam/read,encoder_out/fw/gru_cell/candidate/bias/Adam_1/read,encoder_out/bw/gru_cell/gates/kernel/Adam/read,encoder_out/bw/gru_cell/gates/kernel/Adam_1/read,encoder_out/bw/gru_cell/gates/bias/Adam/read,encoder_out/bw/gru_cell/gates/bias/Adam_1/read,encoder_out/bw/gru_cell/candidate/kernel/Adam/read,encoder_out/bw/gru_cell/candidate/kernel/Adam_1/read,encoder_out/bw/gru_cell/candidate/bias/Adam/read,encoder_out/bw/gru_cell/candidate/bias/Adam_1/read,train_op/control_dependency,save/control_dependency,init_ops/report_uninitialized_variables/boolean_mask/GatherV2,Merge/MergeSummary,train_step/GreaterEqual,train_step/Equal