image2mass

[HHLiufighting]

Hi, I'm Liuhonghao.

I am very interested in your research, but I have a question. How do you obtain the thickness mask? I am very confused about this. Can you provide the code to obtain the thickness mask corresponding to each picture?

Thank you very much in advance,
Liuhonghao
email: [liuhonghao_q@foxmail.com]

[tstandley]

Thanks for your interest!

You can run the entire pipeline with predict_mass.py. This will extract the thickness mask and then run the shape-aware model with it.

If you just want the mask, you can see an example of how to obtain that in model_wrapper.py starting at line 96.

This code is old and needs the proper library versions to run properly. I'm no longer sure what those are, but it's a good bet that things worked in 2017.

[HHLiufighting]

Thanks for your interest!

You can run the entire pipeline with predict_mass.py. This will extract the thickness mask and then run the shape-aware model with it.

If you just want the mask, you can see an example of how to obtain that in model_wrapper.py starting at line 96.

This code is old and needs the proper library versions to run properly. I'm no longer sure what those are, but it's a good bet that things worked in 2017.
Thank you for your reply. I think you misunderstood me. I am talking about the 500K image and thickness mask pair made using the shapenet data set. How to get the true value of the thickness mask? I am confused about this, so I thought Please give me some advice. Of course, it would be better if you could provide the overall training network code.
Thank you very much in advance,
Liuhonghao
email: [liuhonghao_q@foxmail.com]

[HHLiufighting]

Thank you for your reply. I think you misunderstood me. I am talking about the 500K image and thickness mask pair made using the shapenet data set.  How to get the true value of the thickness mask? I am confused about this, so I thought Please give me some advice.  Of course, it would be better if you could provide the overall training network code. Thank you very much in advance, Liuhonghao email: ***@***.*** HAO_L_L ***@***.***  

…

------------------ 原始邮件 ------------------ 发件人: "tstandley/image2mass" ***@***.***>; 发送时间: 2023年10月24日(星期二) 中午1:12 ***@***.***>; ***@***.******@***.***>; 主题: Re: [tstandley/image2mass] image2mass (Issue #7) Thanks for your interest! You can run the entire pipeline with predict_mass.py. This will extract the thickness mask and then run the shape-aware model with it. If you just want the mask, you can see an example of how to obtain that in model_wrapper.py starting at line 96. This code is old and needs the proper library versions to run properly. I'm no longer sure what those are, but it's a good bet that things worked in 2017. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[tstandley]

Unfortunately, I never released these. The main reason is that I simply don't think it was great work and I'm sure someone trying to recreate it in 2023 would be able to do a much better job than I could in 2017. But here's what I did and the issues I had from memory.

1. I downloaded the ShapeNetCore.v2 dataset.
2. I used blender with the python API to render each of these from a random camera position to get the input image (x). This was problematic because blender was constantly dropping textures and the lighting didn't usually look good.
3. I discovered that one of the reasons for the dropped textures was that many of the obj files had two triangles for every triangle and was relying on backface culling to remove the one oriented away from the camera. But turning on backface culling in blender was not possible to do programatically in 2017.
4. I wrote a python script using opengl to "render" each polygon using backface culling in a particular color (particular to the specific triangle). Then I read all the pixels and noted which ones were visible. By doing this hundreds of times per obj file I could figure out which triangles were bad (interior facing) and remove them.
5. I re-rendered the corrected obj files with blender.
6. I also wrote a python script to voxelize the obj files. (get a voxel representation of each object) from the same perspective as the image was rendered. This allowed me to compute the approximate thickness along the appropriate dimension.
7. I tried to find a bounding box (to get bbox dims for each OBJ) for each object. Fitting the minimum bounding box to each object was non-trivial, and my solution wasn't always completely optimal, but it still helped in the end.
8. I put all that data together into a single file.

It would be a pain to get that code or the training code working again in 2023, but if you must have the mask files, I found them in my archive and I can send them to you.

[HHLiufighting]

Unfortunately, I never released these. The main reason is that I simply don't think it was great work and I'm sure someone trying to recreate it in 2023 would be able to do a much better job than I could in 2017. But here's what I did and the issues I had from memory.

1. I downloaded the ShapeNetCore.v2 dataset.
2. I used blender with the python API to render each of these from a random camera position to get the input image (x). This was problematic because blender was constantly dropping textures and the lighting didn't usually look good.
3. I discovered that one of the reasons for the dropped textures was that many of the obj files had two triangles for every triangle and was relying on backface culling to remove the one oriented away from the camera. But turning on backface culling in blender was not possible to do programatically in 2017.
4. I wrote a python script using opengl to "render" each polygon using backface culling in a particular color (particular to the specific triangle). Then I read all the pixels and noted which ones were visible. By doing this hundreds of times per obj file I could figure out which triangles were bad (interior facing) and remove them.
5. I re-rendered the corrected obj files with blender.
6. I also wrote a python script to voxelize the obj files. (get a voxel representation of each object) from the same perspective as the image was rendered. This allowed me to compute the approximate thickness along the appropriate dimension.
7. I tried to find a bounding box (to get bbox dims for each OBJ) for each object. Fitting the minimum bounding box to each object was non-trivial, and my solution wasn't always completely optimal, but it still helped in the end.
8. I put all that data together into a single file.

It would be a pain to get that code or the training code working again in 2023, but if you must have the mask files, I found them in my archive and I can send them to you.

Thank you very much for your patient answer. I think I still hope to refer to your training code. Can you please send it to me while you are busy? It is best to include the code for obtaining the true value of the thickness mask and the training of the entire process. code. I find your work to be very meaningful and interesting. Hope this can be reproduced with your help. I also have a question: Your follow-up work has little to do with assessing the quality of objects. Do you have no plans to continue research in this direction?

Thank you very much in advance,
Liuhonghao
email: [liuhonghao_q@foxmail.com]

[HHLiufighting]

Thank you very much for your patient answer. I think I still hope to refer to your training code. Can you please send it to me while you are busy?  It is best to include the code for obtaining the true value of the thickness mask and the training of the entire process. code. I find your work to  be very meaningful and interesting. Hope this can be reproduced with your help. I also have a question: Your follow-up work has little to do with  assessing the quality of objects. Do you have no plans to continue research in this direction? Thank you very much in advance, Liuhonghao email: ***@***.*** HAO_L_L ***@***.***  

…

------------------ 原始邮件 ------------------ 发件人: "tstandley/image2mass" ***@***.***>; 发送时间: 2023年10月25日(星期三) 凌晨5:58 ***@***.***>; ***@***.******@***.***>; 主题: Re: [tstandley/image2mass] image2mass (Issue #7) Unfortunately, I never released these. The main reason is that I simply don't think it was great work and I'm sure someone trying to recreate it in 2023 would be able to do a much better job than I could in 2017. But here's what I did and the issues I had from memory. I downloaded the ShapeNetCore.v2 dataset. I used blender with the python API to render each of these from a random camera position to get the input image (x). This was problematic because blender was constantly dropping textures and the lighting didn't usually look good. I discovered that one of the reasons for the dropped textures was that many of the obj files had two triangles for every triangle and was relying on backface culling to remove the one oriented away from the camera. But turning on backface culling in blender was not possible to do programatically in 2017. I wrote a python script using opengl to "render" each polygon using backface culling in a particular color (particular to the specific triangle). Then I read all the pixels and noted which ones were visible. By doing this hundreds of times per obj file I could figure out which triangles were bad (interior facing) and remove them. I re-rendered the corrected obj files with blender. I also wrote a python script to voxelize the obj files. (get a voxel representation of each object) from the same perspective as the image was rendered. This allowed me to compute the approximate thickness along the appropriate dimension. I tried to find a bounding box (to get bbox dims for each OBJ) for each object. Fitting the minimum bounding box to each object was non-trivial, and my solution wasn't always completely optimal, but it still helped in the end. I put all that data together into a single file. It would be a pain to get that code or the training code working again in 2023, but if you must have the files I found them in my archive and I can send them to you. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[tstandley]

I should maybe clarify that the Amazon objects don't have ground-truth thickness masks. The thickness model was created with a separate dataset (objectnet). Was that your understanding? If you want inferred thickness masks for the amazon objects, just run the thickness mask computation code that I mentioned above on each object and save the results.

I did find the computed object renders and their corresponding thickness masks which I could easily share. But I think you're better off re-computing these with a more modern 3d object dataset. You have the resulting model already. What are you trying to accomplish?

In terms of the training code, I really don't think you want it. It will be much easier to write it up new in pytorch than to try to get the old code running. It's the type of thing that should be simple, but at the time was very complicated.

Plus, I have released a much better dataset for this paper: https://emma.stanford.edu/. There is 30x more mass data and it is more varied. You should use this dataset if you are trying to produce a good image2mass model.

What are you trying to accomplish? Let me know and I'll help you get there as best I can.

[HHLiufighting]

Thank you very much for your patient answer. I think I still hope to refer to your training code. Can you please send it to me while you are busy? It is best to include the code for obtaining the true value of the thickness mask and the training of the entire process. code. I find your work to be very meaningful and interesting. Hope this can be reproduced with your help. I also have a question: Your follow-up work has little to do with assessing the quality of objects. Do you have no plans to continue research in this direction?

Thank you very much in advance, Liuhonghao email: [liuhonghao_q@foxmail.com]

Hi, I'm Liuhonghao.

Thank you very much for your patient answer. I understand that the real thickness mask is obtained through the shapenet data set, and the ability to predict the thickness mask needs to be pre-trained through the shapenet data set. I wish to reprocess the 3D model for training to obtain the thickness mask. However, it is a bit difficult for me to implement this process by myself. I think my coding ability needs to be improved, so I hope to refer to the code of your pre-trained model, as well as the overall joint training to obtain the training code of object quality. Thank you very much for recommending this better dataset, I think I will use it.

Thank you very much in advance,
Liuhonghao
email: [liuhonghao_q@foxmail.com]

[HHLiufighting]

Hi, I'm Liuhonghao. Thank you very much for your patient answer. I understand that the real thickness mask is obtained through the shapenet data set, and the ability to predict the thickness mask needs to be pre-trained through the shapenet data set. I wish to reprocess the 3D model for training to obtain the thickness mask. However, it is a bit difficult for me to implement this process by myself. I think my coding ability needs to be improved, so I hope to refer to the code of your pre-trained model, as well as the overall joint training to obtain the training code of object quality. Thank you very much for recommending this better dataset, I think I will use it. Thank you very much in advance, Liuhonghao email: ***@***.*** HAO_L_L ***@***.***  

…

------------------ 原始邮件 ------------------ 发件人: "tstandley/image2mass" ***@***.***>; 发送时间: 2023年10月31日(星期二) 凌晨3:22 ***@***.***>; ***@***.******@***.***>; 主题: Re: [tstandley/image2mass] image2mass (Issue #7) I should maybe clarify that the Amazon objects don't have ground-truth thickness masks. The thickness model was created with a separate dataset (objectnet). Was that your understanding? If you want inferred thickness masks for the amazon objects, just run the thickness mask computation code that I mentioned above on each object and save the results. I did find the computed object renders and their corresponding thickness masks which I could easily share. But I think you're better off re-computing these with a more modern 3d object dataset. You have the resulting model already. What are you trying to accomplish? In terms of the training code, I really don't think you want it. It will be much easier to write it up new in pytorch than to try to get the old code running. It's the type of thing that should be simple, but at the time was very complicated. Plus, I have released a much better dataset for this paper: https://emma.stanford.edu/. There is 30x more mass data and it is more varied. You should use this dataset if you are trying to produce a good image2mass model. What are you trying to accomplish? Let me know and I'll help you get there as best I can. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>