I've mentioned here on several occasions that I've been doing a UROP regarding nanophotonics/photonic crystals. Specifically, my first project was in determining whether particular types of photonic structures might enhance absorptivity of light, which would help solar cells that convert that light into electricity. The goal is that the absorptivity enhancement (versus no texturing over the solar cell) should be over as broad of a frequency band as possible, because it is difficult to manufacture many different kinds of photonic structures just to satisfy performance demands over many narrow frequency bands. I worked on this project for about 4 months, because that was the time that my postdoctoral UROP supervisor was around (he moved after that). I was able to get some really nice-looking results in that time, and I figured there wasn't much more I needed to do to wrap it up, so I felt comfortable generally moving on to a new project (which I have been working on since 2012 February or so). The enhancement results looked great compared to existing designs, so I thought we might be on to something here. Recently, things started gearing up for a publication submission.
Today, it all came crashing down. Why? Another postdoctoral UROP supervisor (who I have worked with since last year primarily on my more current project but recently joined in to help progress of the older project, which is the subject of this post) asked me some hard questions about what I was really doing. Because of this, I realized that one of the parameter choices in my calculations that were giving such nice results was fatally flawed. When I fixed that issue, the results I was getting suddenly looked significantly less compelling; with that, any dreams of publication were dashed.
Why did this happen? It boils down to me not being skeptical enough about what was going on. A large part of this has to do with the fact that because this was my first UROP, I didn't have a great idea of what was going on in terms of details. And because that time I spent was only about 4 months and was followed immediately by a new project, I didn't spend much more time on that project after that. Ultimately I got complacent in more ways than one. Because the code I was using was based on existing code for similar calculations, I figured it must have been written to work even with the modifications I was making. I also figured that because I had been getting consistently good results from what I had done over those 4 months, I just needed to worry about those results on the surface and not the fundamentals operation of the code. Those two assumptions combined such that even though I had seen the results of not being careful in my second UROP project and had become much more careful about checking that code as a result, I didn't think I needed to apply the same level of care in checking the code used for the first project.
After realizing the implications of this, I did a few more calculations in a significantly more mopey mood. But then I thought about this and I realize that I shouldn't feel so bad about this. Why is that? Here are a few reasons in no particular order.
1. I've made similar mistakes before, and I've really come to learn from them. One example of something that I thought was going great but turned out badly has to do with email. People who know me may have heard this story, and people who have known me for a while may have actually been there to see me do this, but I won't share the story now; I believe it is sufficient to say that I am now a lot more careful when sending emails especially to large groups of people. A reverse example actually comes from my second UROP project: for a while I was making a mistake in my code that was giving garbage, but after many months of trying various fixes, one particular fix solved all the other issues. Since then I have been a lot more careful about checking my code for that project (though I guess I was confident enough about the code used in the first project that I thought such a high level of care might be unnecessary).
2. I didn't think I would be in the position of having my work for both projects on paper until very recently. Now I can go back to thinking that in any case my second project work would be more likely to go on paper (especially as I know that I have taken a lot more care in checking my code for that project).
3. Several months ago, when my second project was stalling, I was asking myself why it wasn't going as smoothly or quickly as my first project. Now I know that the first project should have in fact gone as slowly as the second project for the work to become as solid and carefully checked. The other part of this issue is that I have been working on the second project continuously, so I have been able to make continuous adjustments to the code and work progressively higher levels of care in checking the code in a smooth manner. Because I essentially stopped working on the first project after those first four months, if I adopt more careful code-checking now, it'll feel more like I'm starting over from scratch, which makes the process feel a lot more frustrating.
4. With all this, I feel like I have already learned a lot more from this lesson than I would have if everything was fine and dandy and this work did get submitted for publication.
5. If nothing else, I hold out hope that I may be able to salvage some good results with the fixes I have made to the code of the first project.
There are two morals to this story. The first is that I shouldn't just check the code I run; I should check it in an actively skeptical manner, always questioning each and every line. The second is that C++ is way more painful to read and (to a lesser extent) write than Scheme is for the kinds of calculations I run.
Today, it all came crashing down. Why? Another postdoctoral UROP supervisor (who I have worked with since last year primarily on my more current project but recently joined in to help progress of the older project, which is the subject of this post) asked me some hard questions about what I was really doing. Because of this, I realized that one of the parameter choices in my calculations that were giving such nice results was fatally flawed. When I fixed that issue, the results I was getting suddenly looked significantly less compelling; with that, any dreams of publication were dashed.
Why did this happen? It boils down to me not being skeptical enough about what was going on. A large part of this has to do with the fact that because this was my first UROP, I didn't have a great idea of what was going on in terms of details. And because that time I spent was only about 4 months and was followed immediately by a new project, I didn't spend much more time on that project after that. Ultimately I got complacent in more ways than one. Because the code I was using was based on existing code for similar calculations, I figured it must have been written to work even with the modifications I was making. I also figured that because I had been getting consistently good results from what I had done over those 4 months, I just needed to worry about those results on the surface and not the fundamentals operation of the code. Those two assumptions combined such that even though I had seen the results of not being careful in my second UROP project and had become much more careful about checking that code as a result, I didn't think I needed to apply the same level of care in checking the code used for the first project.
After realizing the implications of this, I did a few more calculations in a significantly more mopey mood. But then I thought about this and I realize that I shouldn't feel so bad about this. Why is that? Here are a few reasons in no particular order.
1. I've made similar mistakes before, and I've really come to learn from them. One example of something that I thought was going great but turned out badly has to do with email. People who know me may have heard this story, and people who have known me for a while may have actually been there to see me do this, but I won't share the story now; I believe it is sufficient to say that I am now a lot more careful when sending emails especially to large groups of people. A reverse example actually comes from my second UROP project: for a while I was making a mistake in my code that was giving garbage, but after many months of trying various fixes, one particular fix solved all the other issues. Since then I have been a lot more careful about checking my code for that project (though I guess I was confident enough about the code used in the first project that I thought such a high level of care might be unnecessary).
2. I didn't think I would be in the position of having my work for both projects on paper until very recently. Now I can go back to thinking that in any case my second project work would be more likely to go on paper (especially as I know that I have taken a lot more care in checking my code for that project).
3. Several months ago, when my second project was stalling, I was asking myself why it wasn't going as smoothly or quickly as my first project. Now I know that the first project should have in fact gone as slowly as the second project for the work to become as solid and carefully checked. The other part of this issue is that I have been working on the second project continuously, so I have been able to make continuous adjustments to the code and work progressively higher levels of care in checking the code in a smooth manner. Because I essentially stopped working on the first project after those first four months, if I adopt more careful code-checking now, it'll feel more like I'm starting over from scratch, which makes the process feel a lot more frustrating.
4. With all this, I feel like I have already learned a lot more from this lesson than I would have if everything was fine and dandy and this work did get submitted for publication.
5. If nothing else, I hold out hope that I may be able to salvage some good results with the fixes I have made to the code of the first project.
There are two morals to this story. The first is that I shouldn't just check the code I run; I should check it in an actively skeptical manner, always questioning each and every line. The second is that C++ is way more painful to read and (to a lesser extent) write than Scheme is for the kinds of calculations I run.