Saturday, April 7, 2012

4/2 - 4/6 CS373 Blog Post

Good evening everyone.

This past week was fairly busy with the World Crises part two project.  A lot of work was put into our project from all of our members, and the results turned out really well!  The website interface has been improved thanks to Matt; the design looks a lot more sophisticated.  With Garrett's help on the changes to the import script, and Andy's help on export, we now have a fully functional importer and exporter that takes in XML data that conforms to the newest, standard schema.  Fahad kept us sane with unit and acceptances tests for the Google App Engine and our scripts, and Wes did an outstanding job with the initial paper, along with the GAE and UML design charts.  As for me? I worked alongside Matt working with the Django templates and primarily with pulling data out of the Google App Engine.  Our team is working really well.  As a matter of fact, we actually got our project finished up by Monday.  Everyone has their primarily responsibility, but we still understand each other's responsibilities.  It's really amazing, and it's wonderful to know that everyone in the group is willing to commit themselves to do their best on this project!

In class, we had talked in depth about classes, subclasses, and how instance and class methods work.  To be honest, I had never really understood the use of keywords like public, private, static, et cetera.  They never really emphasized those key terms in 307 or 315.  But now it makes much more sense!  In Java projects, I've been used to shamefully throwing 'public static' or any other key term on every function, without really understanding that these are in fact, class methods.  This is such an incredibly grateful thing to have learned, and I'm glad that I now fully understand it.  This actually was helpful on Friday.  I wasn't in class that day because I had an interview with Evernote Corporations.  An interview question they had asked me was to design a brand new framework in a language of my choice of two dimensional geometric shapes.  My idea, in short, was to create a general class, Shape, that had subclasses such as Triangle, Square, Rhombus, et cetera.  These different subclasses would inherit properties by the general Shape class, such as number of points, average length (assuming that all sides were equal in length), and so on.  They had also asked me about making a function that determines the area of a given Shape, and to decide what key words would be included in the function declaration.  I had decided that it should be an instance method because the area of a given Shape requires certain things that can only be conceived with instances.  So, what I had learned this week actually helped in the short run; not just for interviews, but for my complete understanding.

Anyway, it seems like we are also learning a bit of SQL, something which I am pretty familiar with.  Looks like we will be talking more about it next week. I can't wait!

Until next time,

Corey

Sunday, April 1, 2012

3/26 - 3/30 CS373 Blog Post

Good evening all.

Sorry for this blog post being a little bit late.  I've got quite a lot on my hands: some Korean homework, a Network Security and Privacy project due Thursday, this project due Wednesday, 4 films I need to watch for Sociology alongside a 3 page (single-spaced) paper about said films due on Friday.  Not my cup of tea; this will definitely be one of my more stressful weeks in the semester.

Our team is currently pounding away at the WC project.  I think we're on a pretty good page with all that we've done so far.  We've managed to get dynamic pages working and our import and export functions seem to be spewing out correct data.  All in all, our team is moving things in rapid pace and we are doing (in my opinion at least) a pretty good job!  Each person kind of has their own strengths in the project, which is interesting and kind of cool.  It's like each person has a certain subset of tools/code that they're familiar with using/developing and they're able to mesh their stuff with everyone else's stuff, and it just works.  I've never really worked in a small developmental group as this, but it is pretty awesome to see how everyone can come together and put in their two cents and create something pretty darn awesome.  I could only imagine how tedious this project would be if it was assigned per individual!

About class: we're getting into some pretty in-depth stuff.  I haven't been very clear with the whole class in a class idea throughout college.  I actually didn't know that static variables declared in an outer class have full visibility to sub classes.  It makes sense, but I never really thought about that; I guess because none of the programs I've ever coded in class ever used sub-classes.  It's interesting but also difficult to wrap my head around.  I guess it's just something I need to mess with myself in order to fully understand it.  Also, I'm not a huge fan of Python's unusual syntax for private variables.  I mean, it makes sense and all; I just think that they could have come up with a better macro than just two underbars.

We're also getting into constructors and deconstructors.  Honestly, I've never dealt with them in Java.  Mainly in C++ when I was learning it.  I think of them as creating properties and data for variables as they are being instantiated, and then properly removed when their deconstructor is called.  I never really understood the deconstructor, mainly because I never implemented anything in them.  I can only assume we will talk about something like that later on, though.

Well, this is it for now.  I must get some rest! It'll be a busy day tomorrow.

Until next time,

Corey

Saturday, March 24, 2012

3/19 - 3/23 CS373 Blog Post

Good evening all!

What a busy week it has been! Dealing with the project and then preparing for an exam shortly thereafter was not a very fun turn-around.  But the week is over, and I guess that might as well be the only good to have come out of this week!

This past week, my team and I worked our butts off on project 4: World Crises, Phase 1.  This website is a collection of crises, people, and organizations that all will eventually be aggregated through the use of a Google App Engine (GAE).  For this phase of the project, all we had to do was to collect data on our items and display them in static pages.  While we were working on this, we were also supposed to get the back-end side to be able to import XML files given an XML schema, and to be able to export data stuffed into the datastore into an XML format.  The static pages and collecting data weren't very difficult; our importer and exporter took some time to develop, though.  We learned that such scripts are very nit-picky and like to take things in at a certain order.  When we successfully figured out what order we wanted things in, we had to make our exporter export the material in the same pattern.  It was more of a nuisance than anything, but we have things working!  Things are linked in the datastore to prevent duplicates via a many-to-many relationship.  This basically means that certain items may share a certain attribute one or more times.  It eliminates redundant information that is duplicated, because no one wants that, you know?  Anyway, the project seems to be not too bad, and our team is pretty awesome right now.  I have pretty positive vibes with everyone and we gel pretty well! It's gonna be really promising when we complete this project.

As far as the test was concerned, it was a bit difficult for me.  I thought I knew the material fairly well, but there was some questions I wasn't prepared for.  As Downing had mentioned in class about being prepared for the programming questions, I hadn't really prepared for them, so I suffered from that aspect of the exam.  I also didn't do as well as I had hoped on the multiple choice aspect.  My Z score for the exam is still positive, but I felt like I could have done a lot better job.  But, things like that happen and sometimes you don't always do the best all the time.  So for the next exam, I'll try to compensate for my low grade!

I'm really wondering what's in store for the next part of the project.  My thoughts are that we are going to make one Python page that regulates what will be displayed depending on what the user wants to view.  We'll also probably incorporate the linking between the GAE datastore and the website to allow for other pieces of data to be displayed on the website.

Well that's it for now, I need to catch up on sleep.

Until next time,

Corey

Tuesday, March 13, 2012

3/5 - 3/9 CS373 Blog Post

Good evening everybody.

At last, we are finally in Spring Break! Although it is great to not have to deal with school for a full week, it is kind of unfortunate that we have to work on a project.  Let alone, we have to work on this project with 5 other people, hoping that all of our schedules will match up properly.

The project seems like it'll be an interesting project to work on, but one that will have to be worked on very slowly since we are in groups with more than 2 people.  Since we all (or at least most of us) haven't had experience using the Google App Engine or ElementTree, we'll have to research and work on it together in order to be fluent with it.  I've been taking a look at it myself for a little bit, and while it doesn't seem too bad, we definitely will have to do some decent collaboration in order to make any headway into the project.  So far, it seems like all of us will be available within the second half of spring break; more notably the weekend following. We'll have a lot of cramming to do in order to get this project done by next Wednesday! On top of that, we have an exam next Friday.  What seemed like a nice vacation now just seems like a lot of work being pushed off until next week!

In class last week, we discussed the topic of overloading functions and overloading generics.  I had basic knowledge of how that was used, and it seems like most of us use it subconsciously when writing out functions.  I tend to design functions to support a decent amount of data structures.  Maybe not every single kind, because then that just means I may have to re-consider the complete structure of my program.  In Java, it's really a pain because of how *gross* the syntax really looks.  Compared to Python, it is rather disgusting, but I suppose necessary if we want functions to be supported by a variety of data types.  I mean, comparing

private static <T extends Comparable<? super T>> T max2 (T x, T y)


to


def my_max (x, y)



is a definite difference. Since Python is typeless too, this helps in dealing with possible types that are to be passed in as parameters.


It'll be an interesting end to the break to see how things progress with the project. We'll see what happens! Hope everyone is enjoying their break.


Until next time,


Corey

Sunday, March 4, 2012

2/27 - 3/2 CS373 Blog Post

Good evening all,

One week away until Spring Break!  Unfortunately it comes with a long and enduring road for me.  I have a project and four midterms that are in my way until then!

This week was mainly about the Netflix project.  I wrote quite an extensive wiki of it (viewed here) detailing all of my algorithms that I tried testing, along with the problem itself.  The basic idea of the Netflix project was to achieve a root mean squared error (RMSE) below Netflix's score, which was around .9474.  The root mean squared error calculated a margin of error between a prediction of ratings (that I would generate based on data) and the actual ratings that users gave movies.  Sifting through roughly 1.5 GB of data, I compiled some useful caches to be used in my program.  The two main caches contained an average rating for each movie, and an average rating for each user.  With these two caches, I was able to predict a movie rating based on what was sent to me in the probe file (this file contained a list of movies whose ratings that needed to be predicted).  This project was not very hard to implement; the hardest part was tweaking the program and looking at what gave a better RMSE.  I had stuck with taking the average movie rating and average user rating and figuring out how to implement these two to come up with a semi-decent prediction.  I ultimately decided to make another cache: one that had the calculations of all movie ratings combined and all user ratings combined.  I had then calculated some offsets based from the average and multiplied a given weight to some of the numbers in order to find a decent rating.  All of this is explained in heavy detail in the wiki, linked above!

We had also talked about some more interesting things about Python, notably how many types inherit a lot of the same ideas.  Strings, tuples, and lists are all able to work on things identically when a string is passed in their constructor.  It's a really interesting idea and one that makes sense.  I like how everything in Python is very succinct for the most part. Although, now I often catch myself writing code in my other classes and I forget to add semi-colons, brackets, and the likes. Honestly, this class is making me look down at Java and look up at Python more and more, every day.  And I don't mind that!

I'm very curious as to how this new project is going to unfold.  I'm excited to see what all we need to do to make a nice looking website, and it'll be fun working with a big group.  It makes me wonder how we'll use Google Code and Python to build an aggregation-based website.

Well, I have lots to do right now, but next week should be a much more extensive blog post.

Until then,

Corey

Sunday, February 26, 2012

2/20 - 2/24 CS373 Blog Post

Good evening everyone,

Another week has passed, and we are getting closer and closer to a glorious Spring break! Anyone have any plans?

In class this week, we had a great presentation by some employees of Mutual Mobile, a mobile software developer company.  They had discussed what they do, how they do it, and why they do it.  As a curious mobile application developer, I've always found it interesting creating apps for mobile technology.  As they had said in their presentation, mobile development is becoming a lot more prominent with the rise in mobile technology.  The influx of smartphone holders has allowed for the birth of mobile development. It's truly an exciting area of development.  Even when I'm busy with projects in multiple classes, I find myself wanting to develop for my iPhone because it's a lot of fun.  I'm so used to developing things in the command line; it's nice seeing some GUI output!  It was also very cool to see the technologies they use.  I had never heard of Jenkins or Jira until they had talked about it; I need to take a look and see if I can incorporate these things into any of my personal projects.

We had also started discussion about the Netflix project, which is due this coming Wednesday.  I had heard about the "Netflix question" about two years ago in my 336 class, but that was more so figuring out the logical aspect of the question; not so much figuring out an algorithm that could beat theirs.  It's been an interesting problem at the least, and there are a lot of things to take into consideration when trying to produce a better RMSE (Root Mean Squared Error).  For instance, the plethora of different caches one could try to implement given the movie data.  We had discussed two in class: providing the average rating for a given movie, and providing the average rating for a given user.  I'm currently in the process of creating these caches to test out in my program (the average rating for a given user does seem to take a while to create, given that we have 1.5GB of files to look at!).  But other ones that can be implemented may be a bit tricky to create.  For instance, finding users who have similar tastes in movies would take a while to look at, unless we developed another meta-cache that could tally up all the different movies one user has seen, and then compare from there.  Another cache is creating an average rating for movies that were created in a certain decade.  For instance, we would have movie averages from the 1950-1960s, 1960-1970s, etc.  We could also go further and develop another cache that had movie averages based on the year a user reviewed it.  So many options to discover; the only downside is actually parsing all of the 1.5GB of data into a useable cache.  But, the rest is fairly easy.

This class has been a great experience; probably one of my favorite classes.  I have learned so much simply through practice.  It wasn't until this class that I became devoted to creating multiple unit tests to assert that my programs work correctly, and it's kind of a crazy thought.  I've looked back at some older programs and projects that I've worked on, and it's absolutely insane that I didn't use unit testing on them!  Unit testing in Java was emphasized a little in 337 (Theory in Programming Practice) but they never emphasized on what we were to test on.  We would test methods for some corner cases, but not so much the fact that it produces valid output all the time, that we can throw/catch appropriately given the potential input we could face, and so on.  It's a little ridiculous.  Unit testing, I've learned, keeps me pretty sane about my programs!

This is all for now, I've got to get working on multiple projects.  More on the Netflix project will come next week!

Until then,

Corey

Saturday, February 18, 2012

2/13-2/17 CS373 Blog Post

Good evening everyone!

It's been a busy week!  The PFD project was due and we had our first exam in the course.  A little stressful, but I think we all survived!

This week, the primary focus was on the Project File Dependencies project.  The basic concept of this project was to take in an input (essentially a graph of nodes that contained a list of dependencies) and output a result where numbers would be placed in a way so that the number before it did not depend on it.  This definitely was a project that could have used several different implementations to get the job done right.  In collaboration with my partner, we ended up choosing the easy route by using a list that would, through iterations, sort itself (with as few calls to sort as possible).  In our program, we started off by reading the input and we read the appropriate amount of lines based on the number we obtained from the first line.  The first number designated how many vertices (or nodes, if you prefer that) are in our graph, while the second number represented how many dependency rules to follow.  We initialized a list filled with empty lists, with the number of empty lists being dependent on the number of vertices that would be in the graph.  We then iterated through the rest of the input to determine the rules.  Each line contained the node number, followed by the number of dependencies it has and the dependency node numbers following that.  The construction of our graph rules ended up appending such data into the graph's corresponding index of the rule's node.  So, for instance, a rule that contained "5 3 1 2 3" would create a list [1, 2, 3] that would fit in graph[4].

Once we filled up our graph, we went down through the list and appended the indices whose dependency lists were empty.  These indices would be appended into an intermediate container (in our case, a sorted list).  Since we would go down our graph every time, the list would be sorted initially.  For every index we found to be empty, we replaced the empty list with a -1 delimiter to signify that it has been "removed" from our graph.  Then, for each node in our intermediate list, we would search through all non-empty dependency lists and remove that node from the dependency list to again signify that it has been removed.  After one run, we add the node to a result list.  We search again and find for any empty lists; if there are, we add them to the intermediate container and sort.  Rinse, repeat, profit.

Through observation, I understand that we could've done a better job in optimizing our code. Every time we add new nodes to our intermediate list, we sort, which can take a while depending on how big the graph is, and how many nodes become free through one removal.  I did learn through research though, that Python's sort() function is an adapted merge sort which runs a lot faster than a traditional merge sort algorithm.  Regardless, had we had used a different kind of container like a linked list or a binary heap,  we could have been more efficient with more optimal code.  Maybe I'll experiment with that later.

In class, we discussed about iteration, parallel assignments, argument keywords, default arguments, unpacking iterable items, and creating tuple and dictionary containers in functions (phew, so many topics!).  I guess it never really hit me, but it makes sense that within a for each loop that items are copied.  It abides to the iterablility (not really a word, but bear with me) standard.  Most of our conversation this week revolved around some interesting quirks with Python.  Parallel assignment was interesting in the sense that one could pass in any iterable object of a given length (designated in the parameters of the function), and it would work.  I guess that's one amazing but potentially dangerous thing about Python; you can pass in just about anything, but it has to make sense.  A list, tuple, set, dictionary, xrange, can all be passed in the example slide with the same value.  Pretty ridiculous.  Passing in arguments as keywords was also a really cool thing to learn about; as Professor Downing had mentioned, this is incredibly useful for defining what your variables are exactly in method calls.  Someone looking at fresh code could see a function call like this:

custom_hash(24, 189, data, "x381jfs")

and would probably have no idea what it meant, other than it uses some kind of data to create some hash.  If instead, it could be re-written as such:

custom_hash(data_begin_index = 24, data_end_index = 189, list_of_data = data, salt = "x381jfs")

one could then decipher that this is a hash function that takes a list that's only interested in a range of the list, and then salts it.  Of course, it is very fragile.  It can be easily broken if one decided that the parameter names were either too ambigious, too long, unnecessary, etc.  Still a useful thing to use.  You can also put the assignments in different places and it would still spit out the right answer, so long as you provide the necessary arguments in the right place for those that aren't assigned explicitly.  Furthermore, you can provide default values for arguments if they aren't passed.  Again, pretty useful if a function call is used several times and a fixed value is used consistently.

We also discussed about unpacking items, as well as creating tuples and dictionaries straight from the parameter line.  Unpacking items was kind of cool.  You could pass in just about any iterable item (like the parallel scheme discussed about above) of a given length, based on the parameter, and it would unpack the data inside to whatever you desire.  That is, it would strip the data from a container to be used for other purposes.  This helps with removing the need to iterate through data and add it to some kind of container.  Alongside of unpacking, we can do the opposite with creating tuples and dictionaries.  Again, useful functions that eliminate those extra lines of codes because programmers can be awfully lazy sometimes.

Lastly, we had a test last Friday.  It didn't turn out as bad as I had expected!  As Professor Downing had said, he geared the test to make the cheat sheet virtually useless, but it did indeed help as a very helpful review over all the material we talked about thus far.  I stumbled a bit on the Haskell problem (I didn't really look into it that much; I had bet that it would've been a multiple choice question!) but I had a rough idea of the syntax.  Hopefully not many points will be deducted from that problem.  All I know is that all of us are anticipating what the final scores will be!

Until next time,

Corey