[Modelzoo] Add serving for DIEN, DeepFM and WDL. by Zhang-Zhenning · Pull Request #319 · DeepRec-AI/DeepRec

Zhang-Zhenning · 2022-07-15T09:33:23Z

I have finished exporting savedmodel for models in modelzoo. Under Deeprec/modelzoo there is a Get_SavedModel.md for new user to learn how to export and where the result model will locate.

CLAassistant · 2022-07-15T09:33:29Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

shanshanpt · 2022-07-17T10:09:26Z

for #232

shanshanpt · 2022-07-17T10:10:43Z

modelzoo/BST/pb_to_pbtxt.py

@@ -0,0 +1,13 @@
+from tensorflow.python.saved_model import loader_impl
+from tensorflow.python.lib.io import file_io
+from tensorflow.python.platform import tf_logging as logging


useless file.

shanshanpt · 2022-07-17T10:14:23Z

modelzoo/BST/prepare_savedmodel.py

@@ -0,0 +1,738 @@
+import time
+import argparse
+import tensorflow as tf


不需要修改原文件，在原文件上添加代码即可。

shanshanpt · 2022-07-17T10:16:01Z

modelzoo/BST/start_serving.cc

+#include "serving/processor/serving/processor.h"
+#include "serving/processor/serving/predict.pb.h"
+
+static const char* model_config = "{ \


除fake数据之外，需要根据实际的train data生成serving data，然后能够根据serving data进行process

shanshanpt · 2022-07-17T10:20:24Z

CLA required.

Zhang-Zhenning · 2022-07-25T09:45:27Z

I have finished exporting savedmodel, extracting data from the input file to generate requests and serving for DIEN DeepFM WDL under modelzoo/features/EmbeddingVariable, please check.
One thing that needs to be emphasized for models with non-string inputs like DIEN, is the model_instance.cc will automatically run CreateWarmup to accelerate real serving, which is inappropriate under cases such as the input placeholders don't have a fixed length. During my local test I comment it out, please contact me if there is any problem.

shanshanpt · 2022-07-25T11:30:07Z

modelzoo/features/EmbeddingVariable/DIEN/generate_data.py

+cat_voc = os.path.join(data_location, "cat_voc.pkl")
+
+def prepare_data(input, target, maxlen=None, return_neg=False):
+    # x: a list of sentences


DeepRec中一般使用两个空格锁进，这些文件都稍微改一下

shanshanpt · 2022-07-25T11:33:05Z

modelzoo/features/EmbeddingVariable/DIEN/generate_data.py

+
+
+f.close()
+


多余的空格删除

shanshanpt · 2022-07-25T11:42:17Z

modelzoo/features/EmbeddingVariable/DIEN/start_serving_dien.cc

+  ::tensorflow::eas::ArrayShape array_shape;
+  ::tensorflow::eas::ArrayDataType dtype_f =
+            ::tensorflow::eas::ArrayDataType::DT_FLOAT;
+  int num_elem = (int)cur_vector.size();


不用(int)强转，会隐式转换的，或者定义为 size_t num_elem也可以

shanshanpt · 2022-07-25T11:43:15Z

modelzoo/features/EmbeddingVariable/DIEN/start_serving_dien.cc

+  int num_elem = (int)cur_vector.size();
+
+  array_shape.add_dim(1);
+  if((int)cur_vector.size() < 0){


if((int)cur_vector.size() < 0) ，cur_vector.size()不会小于0的，类型是size_t本身是正值。

shanshanpt · 2022-07-25T11:44:15Z

modelzoo/features/EmbeddingVariable/DIEN/start_serving_dien.cc

+
+  return input;
+  }
+  array_shape.add_dim((int)cur_vector.size());


cur_vector.size()前面已经赋值了num_elem

shanshanpt · 2022-07-25T11:57:49Z

modelzoo/features/EmbeddingVariable/DeepFM/train.py

    if tf_config:
        print('train steps : %d' % train_steps)
+        print("-----------")
+        print("-----------")


useless code

shanshanpt · 2022-07-25T11:59:15Z

modelzoo/features/EmbeddingVariable/SavedModel_Serving.md

@@ -0,0 +1,58 @@
+## How to use prepare_savedmodel.py to get savedmodel


文件名叫README.md就可以

shanshanpt · 2022-07-25T12:00:22Z

modelzoo/features/EmbeddingVariable/WDL/start_serving_wdl.cc

+
+}
+
+::tensorflow::eas::ArrayProto get_proto_f(float char_input,int dim,::tensorflow::eas::ArrayDataType type){


函数重复

shanshanpt · 2022-07-25T12:00:58Z

modelzoo/features/EmbeddingVariable/WDL/start_serving_wdl.cc

+
+          while (record != NULL) {
+              // only 1 label and 39 feature
+              if (j >= 40) break;


shanshanpt · 2022-07-25T12:01:11Z

modelzoo/features/EmbeddingVariable/WDL/start_serving_wdl.cc

+        struct input_format39 inputs;
+        inputs.I1 =  (float)(atof(all_elems[start_idx]));
+        inputs.I2 =  (float)(atof(all_elems[start_idx+1]));
+        inputs.I3 =  (float)(atof(all_elems[start_idx+2]));


这个搞一个循环吧

shanshanpt · 2022-07-28T02:24:31Z

先删除掉不必要的修改。

shanshanpt

先把所有的文件整理一下

shanshanpt · 2022-07-28T12:17:18Z

modelzoo/features/serving/README.md

+## How to use prepare_savedmodel.py to get savedmodel
+
+- Current support model: \
+BST, DBMTL, DeepFM, DIEN, DIN, DLRM, DSSM, ESMM, MMoE, SimpleMultiTask, WDL


Now support DIEN, DeepFM and and WDL.

shanshanpt · 2022-08-08T02:12:57Z

modelzoo/features/serving/DIEN/generate_data.py

+                        maxlen,
+                        data_location=data_location)
+
+f = open("./test_data.csv","w")


test_data.csv 怎么得到的呢，可以在readme中写清楚

shanshanpt · 2022-08-08T02:16:04Z

modelzoo/features/serving/DIEN/prepare_savedmodel.py

+
+    with tf.Session() as sess1:
+
+        model = Model_DIN_V2_Gru_Vec_attGru_Neg(n_uid, n_mid, n_cat,


这个Model_DIN_V2_Gru_Vec_attGru_Neg是哪里import的？

shanshanpt · 2022-08-08T02:17:50Z

modelzoo/features/serving/DIEN/start_serving_dien.cc

+  input.set_dtype(dtype_f);
+
+  switch(dtype_f){
+      case 1:


两个case中大部分代码都在重复的，可以只在
input.add_float_val((float)atof(cur_vector.back()));
input.add_int_val((int)atoi(cur_vector.back()));
代码加上if判断。

shanshanpt · 2022-08-08T02:33:58Z

modelzoo/features/serving/DIEN/start_serving_dien.cc

+          temp_ptrs.clear();
+
+          // traverse current line
+          record = strtok(line, delim);


下面这个可以搞成类似 split(...) 函数吗，现在这样写比较hack。

shanshanpt · 2022-08-08T02:39:09Z

modelzoo/features/serving/DeepFM/start_serving_deepfm.cc

+
+        // input setting
+        ::tensorflow::eas::ArrayProto I1 =  get_proto_cc(&inputs.I1_13[0],1,dtype_f);
+        ::tensorflow::eas::ArrayProto I2 =  get_proto_cc(&inputs.I1_13[1],1,dtype_f);


搞一个数组存储？

shanshanpt · 2022-08-08T02:39:51Z

modelzoo/features/serving/DeepFM/start_serving_deepfm.cc

+int main(int argc, char** argv) {
+
+  // PLEASE EDIT THIS LINE!!!!
+  char filepath[] = "/home/deeprec/DeepRec/modelzoo/features/EmbeddingVariable/WDL/test.csv";


文件如何生成的呢

shanshanpt · 2022-08-08T02:41:04Z

modelzoo/features/serving/README.md

+ For every model listed above, there is a prepare_savedmodel.py. To run this script please firstly ensure you have gotten the checkpoint file from training. To use prepare_savedmodel.py, please use:
+
+  ```
+    cd [modelfolder] 


training的文件位置需要写清楚，用户可能找不到具体的training的文件所在。
或者你把training文件copy到你的目录下也可以。

shanshanpt · 2022-08-08T02:41:51Z

modelzoo/features/serving/WDL/start_serving_wdl.cc

+int main(int argc, char** argv) {
+
+  // PLEASE EDIT THIS LINE!!!!
+  char filepath[] = "/home/deeprec/DeepRec/modelzoo/features/EmbeddingVariable/WDL/test.csv";


文件如何生成的呢

shanshanpt · 2022-08-08T02:42:10Z

modelzoo/features/serving/WDL/start_serving_wdl.cc

+            ::tensorflow::eas::ArrayDataType::DT_INT64;
+
+        // input setting
+        ::tensorflow::eas::ArrayProto I1 =  get_proto_cc(&inputs.I1_9[0],1,dtype_f);


搞一个数组存储

gumanchang · 2023-10-25T09:47:04Z

请问下，这个代码还是没有合并嘛？现在仍然只有训练，没有推理部署，文档也没说

candyzone · 2023-10-25T10:01:26Z

请问下，这个代码还是没有合并嘛？现在仍然只有训练，没有推理部署，文档也没说

DeepRec 本身具备推理的能力，这个PR只是这三个model 的推理例子，你可以参考

shanshanpt self-requested a review July 17, 2022 10:07

shanshanpt reviewed Jul 17, 2022

View reviewed changes

shanshanpt reviewed Jul 25, 2022

View reviewed changes

shanshanpt reviewed Jul 28, 2022

View reviewed changes

reorganize the file structure

576652e

Zhang-Zhenning force-pushed the inference_zzn branch from b52d097 to 576652e Compare July 28, 2022 08:59

shanshanpt reviewed Aug 8, 2022

View reviewed changes

liutongxuan changed the title ~~savedmodel generation~~ [Modelzoo] Add serving for DIEN, DeepFM and WDL. Aug 11, 2022

		@@ -0,0 +1,58 @@
		## How to use prepare_savedmodel.py to get savedmodel


		}

		::tensorflow::eas::ArrayProto get_proto_f(float char_input,int dim,::tensorflow::eas::ArrayDataType type){


		with tf.Session() as sess1:

		model = Model_DIN_V2_Gru_Vec_attGru_Neg(n_uid, n_mid, n_cat,



		f.close()

Conversation

Zhang-Zhenning commented Jul 15, 2022

Uh oh!

CLAassistant commented Jul 15, 2022

Uh oh!

shanshanpt commented Jul 17, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shanshanpt commented Jul 17, 2022

Uh oh!

Zhang-Zhenning commented Jul 25, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shanshanpt commented Jul 28, 2022

Uh oh!

shanshanpt left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gumanchang commented Oct 25, 2023

Uh oh!

candyzone commented Oct 25, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants