My Thesis - A Retrospective
Intro
Why did I do a thesis in the first place? It was not required and the typical path to a MS in CS at UTD is to just take 11 classes instead of 9 classes and a thesis. There were three reasons that come to mind. First was pride. I wanted something to show when I was done with my MS, something that I accomplished that distinguished me from everyone else who also has an MS. Scheduling was another reason. Being limited to evening classes, the thesis allowed me to finish at least a semester or two earlier than trying to schedule an additional six hours of classes in the evening. Finally, stupidity (aka pride again) was the final reason for choosing the thesis over two additional classes. This was the path less traveled, the more difficult route and obviously I want to take the more challenging path. I wanted to push myself and complete my MS in two semesters, with the last semester consisting of six hours of classes and six hours of thesis while working full time. Challenging? Yes. Smart choice? No.
Now that I have justified my actions, and now that I am done with my thesis what questions can I answer? What do I with I knew before I started? What was easy that might be considered hard? What was hard that might be considered easy? What did I learn from my thesis? Finally, what should I have done differently?
What I wish I knew before I started?
The time commitment When I started, I knew the thesis was going to take a serious amount of my time. I expected to spend 20 hours a week on my thesis for about 5 months solid, starting in late October and working until the middle of April, with most of the real work being done in Jan-Mar and April spent writing the paper. April was going to be a lite month, with much less time being spent on the thesis, with the paper and defense being completed by the middle of April.
This plan started out just fine and everything seemed to proceed as planned, but in March, I noticed that the goal of finishing the research for the thesis kept moving one week further into the future. This continued until the middle of April, with the thesis topic finally solidifying about 10 days before the final draft of the paper must be submitted to the graduate deans office, and the last set of test results were collected only about 5 days before the paper was committed. So the thesis started out needing 20 hours a week, and these were not very stressful hours, but the number of hours needed continued to increase as did the stress level. For 7 of the last 12 days, I slept every other day, and the days I was sleeping, it was only for six hours. This was incredibly unhealthy, and my body finally revolted against my mind, or maybe both revolted against my thesis. After recovering, I decided to take off work until I finished my thesis so I could afford to sleep at night.
Most of the crazy hours I put in were at the very end, and I was ignoring almost everything but my thesis. Once I had results my advisor and I were happy with, I could have filed for In Absentia graduation, which would have gotten me another six weeks to complete my thesis. Being more interested in graduating than anything else at the time, I pushed through, and I can't say I am sure I made the right decision.
The large unknown So what happened with my well planned schedule? Did I slack off? No, the problem is there is a goal, and that goal is likely one that no one has reached before, so there is no good indication of how close you actually are or how much longer it will take to get there. This unknown can be incredibly frustrating. I actually changed direction significantly about 30 days before the due date, looking for an alternative goal that would be achievable in the time left because forward progress had stopped complete for the goal I was attempting to achieve.
More background on NLP When I started my thesis I had a good understanding of some basic machine learning techniques, but I had not taken any natural language processing classes. So not only did I know nothing of NLP, no one really uses the basic machine learning techniques, so I had a lot to learn. This really was a minor problem, that made communication about what I was doing an issue more than anything.
What was easy?
Implementations Surprisingly, the implementations of the algorithms were fairly easy. I completed a broad base of work before focusing on thesis topic, including case restoration, sentence boundary identification and many variations of entity detection. What took the longest to implement was the sentence boundary identification. Starting with the paper MXTerminate is based on, and using MXTerminate as the benchmark it only took two weeks, or about 40 hours, to read and understand the paper, find a Maximum Entropy library I liked and write everything in C++, which is one of my weaker languages. I did write a lot of code, but I didn't spend a lot of time doing it, and most of the code that was written for my thesis was automating processes so they were easily repeatable. Writing code might be something that a CS grad might expect to spend a large portion of time working on, but in my case it was rather minimal.
Writing 60 pages I thought writing the paper would be difficult. The bulk of the paper is quite easy to write. There were sections that I had trouble with, because I didn't really feel I had anything interesting to say about those sections, but there are other sections that are very easy to write, and these tend to be the larger chapters. For my thesis, I felt that the approach was the most interesting aspect, and the chapter on my approach makes up about 50% of the length of the paper. Considering there are about 15 pages of BS (title pages, signatures pages, TOC, list of tables, list of blah blah blah) which really are incidentally generated, all the work went into one chapter. It is probably common for most of the effort of the paper to go into the approach or into analyzing the results.
Coming up with new ideas for enhancements When I started, I didn't really understand my problem that I was working on all that well. Once I had spent about a month working on the problem, I started to grasp why I was using certain features, and why I was using the algorithms I was using. Once I crossed that threshold, coming up with new ideas to try and improve performance was easy. It was so easy in fact, that most of my ideas never made it to a trial, but instead died as just a thought jotted down in my notebook. This segment of coming up with improvements to a working solution and testing them to see if they actually work will fill any available time.
What was hard?
Dealing with the unknown This didn't hit me until late in my thesis, but I was doing research, and neither I nor my advisor knew if what I was attempting was really going to work. My advisor knew it would "work" but I didn't know that he wasn't sure that that it would work well enough to justify writing about. Advisors can be sneaky like that, letting you miss the obvious, and continue forth in ignorant bliss. Not knowing when I was going to be done was very stressful at the end, when I could see deadlines approach and I still did not know when the research phase would be done.
Final draft of the paper So writing the paper wasn't all that bad, but what sucked was finishing it. And I don't mean taking your first draft and turning it into something you and your advisor are happy with, but taking that paper that is technically complete, but still has to pass critical eye of the Graduate Reader, aka the formatting Nazi. Luckily, the formating Nazi is a very nice person, but I walked across campus enough times to develop blisters, and from the time I thought I had the formatting correct, to the time when it was actually correct was about six hours. LaTeX makes many things easy, and I am glad I used it, but some of the formatting is done with the black magic of LaTeX, so that I had one title that was about a quarter of an inch too high on one page. It took a couple of hours to figure out how to move this down the page. This was the day the paper was due, and I finished at 3pm, for a 5pm deadline. It was an unfun experience to say the least.
Coming up with ideas that actually improve performance Coming up with new ideas to try is easy, finding just one that is worth keeping is difficult. Some things are cheap and easy to add in personal time, but take a lot of clock time to test. For example, I could add and new feature in about 30 minutes. Verifying that it was worth adding took at least a couple of hours at the end. In the middle, before I changed how I was handling everything, it could take 3 days. This is right there with the unknown. You have a good idea that you think should work logically, and improve performance, but because you already have a feature with significant overlap, or you were just plain wrong in your assumptions, the addition doesn't improve performance, or maybe decreases performance. It amazed me how much trial and error was needed to find things that actually worked.
What I learned?
Academic work is different from Professional work, but still work Now if I had 10 year of academic experience, my opinion might differ, but it seems that how things get done academically differs from how they get done professionally. It almost might be that I have a lot more control over how things get done professionally and the experience to influence the process and lack that academically. What am I talking about? Nothing seems to get done until the deadline and there is no breaking up a larger task into smaller tasks that can be completed on a schedule. I never felt like I was done with any part of the thesis until the paper was handed in. Several times I thought I was done with the research and I could start on the paper, by the time I actually started on the paper, I had given up on this idea, because I knew there would be results I would want to include in the paper that I hadn't run yet, and this was true. At the end, I was working on the paper non-stop, so the only thing that mattered was having the paper ready before the cut off. There were no breaks or little achievements during that time period, there was no time to think about such things. Earlier on in the research phase, there were definitely tasks that were completed, but these tasks ended up having nothing to do with my thesis, so weren't actually moving the process along. Of course, if you knew what approach would work and what feature would produce the best results and why any variance in the results existed, then a thesis could be completed in three weeks, research, paper and defense, no problem. Working on thesis unknowns is the reason to do a thesis, but to someone who is used to knowing what the end goal is, how to get there and how to track his progress along the way, the thesis process can be a bit maddening.
My Advisor is not what I thought he would be This one really surprised me. I thought I knew my advisor well before my thesis, but you don't really know what working with a person is going to be like until you work with that person. I wanted an advisor that would challenge me, and I got that, but my advisor was subtle about it. Sometimes things need to be obvious, and sometimes even obvious things need to be pointed out multiple times. This is not exactly something that I thought I would need, but it something that would have been helpful in retrospect. Now if you asked me, I could tell you that Dr. Ng has a teaching style where he is intentionally vague with the expectation that you will get it or you will ask for details. This extends to his communication style, where he will hint at a path that he thinks you should take and if you are unsure about the direction that should be taken, you should ask. This is obvious to everyone who has worked with Dr. Ng, but for some reason I expected him to be more definitive. Sometimes I need to obvious pointed out to me. Multiple times.
I don't want to do a PhD I think everyone who is considering a PhD should write a Master's thesis first. MIT actually requires this, and I think it is a very good idea, because you don't really know what you are getting into until you go down this path. Class work is not research, and academic research is not working for a living. I had considered getting my PhD, but after finishing my Master's Thesis, I don't think it is something I wish to pursue. Instead of spending six months doing research and writing a paper, I would spend 3 years? Oh, and I would get paid almost nothing during that time? Sounds great! Or maybe not. I have not discussed this with my advisor, and he probably has an even better idea than I do at this point if I would be cut out to continue. So the research was fun, and I think I would enjoy working on a problem in a field that I was excited about, but I am not sure that I want that to be the focus of my life for several years. It is very easy for me to go home and turn my work life off. My thesis was very much an all consuming experience, and I am not sure I want to go through that in a prolonged experience.
What I should have done
Kept a running progress report Oh man, this is something that burned me several times badly. What I needed was a better way to communicate what I had done and what the results were. My weekly statuses should have included this information, and it should have been easy to track where I was the previous week and where I had gotten to this week. On several occasions there were miscommunication where my advisor thought my performance was higher than it actually was, so when I showed him an improvement at a later date, to him is was a decrease in performance. This lead to me pursuing paths that my advisor, if he had known the actually performance, would have discouraged.
Kept a lab notebook The scientific method is a wonderful thing. It has nice orderly steps, and if you document those steps then it is really easy to explain what you did and why you did it and how the results turned out. Over and over I told myself that I needed to be more organized, and over and over, my advisor would ask me a question during a status meeting I should have had an answer too, but didn't because my notes were weak to non-existent.
Communicate better with my advisor I am sure this is something that is always true no matter how well you do communicate, if you did a little better communicating, time consuming tasks could have been avoided because your advisor would have a better idea what you were working on, and the time you were spending on it. I don't think my advisor has any idea how much time I was spending on my thesis the last six weeks, because I spent so much time working on things that he wasn't aware of. The worst was, he wanted me to implement another algorithm, and I started it, saw that it wouldn't out perform what I was already working on, so I dropped it. Well this new algorithm was a more interesting approach, worth publishing about even with lower performance, and created new situations that could be exploited that would be worth writing about. Since I detoured back to my original track instead of following through with what he asked me to work on, I lost about a week of time. This week was also lost during the critical final month, so it was incredibly painful. If I had to do it again, I would tell my advisor all the stupid things I was trying, and he could have made sure I wasn't wasting too much time. Time wasting is required to do a thesis, but it can be limited intelligently.
Conclusion
Don't do a THESIS! That would be the easiest thing to do, but a thesis is a worthwhile experience. Likely, it will be more work and take more time, and turn out very different then what your expectations were before starting, but for some people, including me, it was a was a very worthwhile experience.
Latest Projects
