Special class time set

The make-up lecture on scientific image processing will be at 3 pm on Monday, May 18. Thanks, George

Scheduling a make-up lecture

Please indicate in this doodle pool all times which wokr for you, for a make-up lecture on scientific image processing:

http://doodle.com/ige7evmh8fngvkd9

Thanks!

Feedback

I agree with what Laura has already posted, but I will evaluate the course from view of someone who perhaps does not use computational science as a core component of her research but would benefit greatly from knowledge of these methods and their casual use. I am in the biochem program and have done some programming in the past, but some of the earlier topics in class may have been a bit too advanced for me. That said, because of my inexperience, the class provided an opportunity to learn a great deal. Lectures on XML, databases, Python, and visualization were especially informative. I spoke with many students who later stopped coming to class citing that the information was too advanced for them even though they had a background in a language like C or Matlab. The wealth of information provided may be more useful to advanced students, but can become intimidating to others.

I would have liked to have seen more demonstrations. It is all fine and dandy to describe the power of these programs and languages, but sometimes without explicit instruction on operation, forays into program use can quickly die. Perhaps a clearly defined set of practice assignments with some constructive feedback might be helpful in getting students started.

Feedback, Neighbourhood Algorithm and an interesting link

Hi all,


Feedback:
---------
First of all I'd like to say that I really enjoyed the class. It is nice to get such a broad overview of topics, some of which I had never realized could be useful for my research as well. So thanks!

One thing that I missed a little is more material on object-oriented programming concepts. I am aware of the fact that this is *not* a programming course, but we touched upon it quite often during several lectures. Maybe this is something for next term?


Neighbourhood Algorithm:
------------------------
At Dr. Jewell's lecture I promised to put some references here concerning the Neighbourhood Algorithm, a direct search technique for global optimization. It is used in geophysics, but I am convinced it can have many uses in other fields as well.

One of the main inventors of this algorithm is Malcom Sambridge at ANU. I had the fortune of taking a class with him when he was a visiting professor in the Caltech GPS department last year.

You can find source code, paper references etc here:
http://rses.anu.edu.au/~malcolm/na/
References to the two original 1999 papers in the 'Papers' section.

Let me know what you think, and if it is useful for you!


Interesting link:
-----------------
I have just started using the computational resources on Teragrid, a national supercomputing network. They have a link to several online courses by the National Center for Supercomputing Applications, on the topics of Parallel Computing, Debugging, and Visualization.
http://ci-tutor.ncsa.uiuc.edu/
Registration is required, but it's all free.


Looking forward to next term!

cheers,
-Laura

We need your feedback!

So the first part of our class is now over! Thank you all for participating, and I hope that you found it at least somewhat useful.

We need and want your feedback:

- What have we done well? What have we done poorly?

- How would you change the class to make it more useful?

- What topics would you like to see addressed in the next term? (see http://www.astro.caltech.edu/~george/aybi199/AyBi199lectures.txt for the current plan - but it can be changed by popular demand)

- Anything else that would improve the class?

Please be frank and constructive. You can be as critical as you want - there will be no reprisals, we are learning as we go, and everyone gets an A or a P anyway. Our goal is to teach you some useful stuff, not to give you a grade.

GGobi: Another Data Explorer

While exploring the links from the Feb. 17 lecture (Introduction to Data Mining), I came across another visualization tool for highly dimensional data that I like very much.

It's not useful for making movies or detailed simulations, but I think it's going to be useful for exploring data sets that are so large you don't even know what you have (or expect!). There are a bunch of extensions to it in R and the website also says that you can extend it in Python and a few other languages.

It's called GGobi and you can find out more about it here: http://www.ggobi.org/

It's open-source and, therefore, free.

Viz 3 Assignment Response - from Toni

I found Correa et al. 2007 (http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=4376157&isnumber=4376125) interesting and directly applicable to what I currently do in the lab. The article explores methods of allowing a viewer to physically manipulate a data set as a means of data exploration. Such probing would be similar to touching and deforming an object with ones hands. The general scheme for deformation involves deforming data points through a displacement map or procedure then sampling these points according to a volume representation. It may be possible to carry out such an operation on data using software such as Volview. The figure to the left demonstrates such a deformation and how it can be useful in highlighting relevant features in a data set. I work with macromolecular structures that might benefit from similar manipulations. For example, atomic resolution structural data of very large protein complexes may be easier to navigate if one can easily deform surrounding atoms to view buried subunits, ligands, metals, etc.

HIV protease with inhibitor

Random Useful References

Some references that might interest people taking this course.

- NetworkX.

A Python package for modelling graphs and networks:

http://networkx.lanl.gov/


- Graphviz.

A graph visualization tool (works with NetworkX):

http://www.graphviz.org/


- A line-by-line Python profiler

A Python profiler that gives you performance information about each line on your code, so you can attack the main bottlenecks first.

http://www.enthought.com/~rkern/cgi-bin/hgwebdir.cgi/line_profiler/summary


- "The visual display of quantitative data", a classic book by Edward Tufte.

http://www.edwardtufte.com/tufte/books_vdqi


- The NEOS server and AMPL

A server that accepts optimization jobs over the web. High-end hardware and high-quality solvers for free! Accepts jobs in the AMPL langugage.

http://www-neos.mcs.anl.gov/
http://www.ampl.com

- The Stony Brook Algorithm Repository

Search for algorithms by problem or by language. The website also provides a ranking of the algorithms (presumably which ones are "best"). For example, I needed an algorithm for matching in bipartite graphs. I just searched under "Graph problems - Polynomial type problems" -> "Matching" and found a list of them, with their ranking.

http://www.cs.sunysb.edu/~algorith/


- "Large Scale Data Analysis Challenges" (talk at Caltech next week)

Tuesday, February 24th
12:00 - 1:00pm
74 Jorgensen

*Lunch will be provided*

SPEAKER:
Dan Meiron
Fletcher Jones Professor of Applied & Computational Mathematics and
Computer Science

TITLE:
Large Scale Data Analysis Challenges

ABSTRACT:
JASON, a scientific advisory group, was asked by representatives of the
Department of Defense (DOD) and the Intelligence Community (IC) to
recommend ways in which the DOD/IC can handle present and future
sensor data in fundamentally different ways, taking into account both the
state-of-the-art, the potential for advances in areas such as data structures,
the shaping of sensor data for exploitation, as well as methodologies for
data discovery.

In this presentation we will examine the challenges associated with the
analysis of large data and in particular compare DOD/IC requirements to
those of several data intensive fields such as high energy physics and
astronomy. The conclusion is that while DOD/IC data requirements are
certainly significant, they are not unmanageable given the capabilities
of current and projected storage technology.

The key challenge will be to adequately empower DOD and IC analysts by
matching analysis needs to data delivery modalities. At a very cursory level,
we will examine some current approaches that could enable better information
fusion. We'll also propose various grand challenges that could be used to
assess and prioritize future research efforts in data assimilation and fusion.

Grading policy

Dear students,

I have clarified with the Registrar that the class can be taken either for a letter grade, or P/F. So, if you are taking it for credit, please email me:

Your name, option, and whether you want a letter grade or P/F

And, just in case you are wondering, the grading will be as follows: I will assume that the honor system is working, and that you have done all of the assignments and suggested exercises and readings. Thus everyone gets an A or a P, and now you can stop worrying about grades, and focus on learning! (In case someone believes that they do not deserve such a grade, please let me know, and I'll give you the grade you think you deserve....)

Thanks, George

Hamming numbers

Hi all,

I was wondering if we could start a discussion on the assignment of Lecture 2 (computing Hamming numbers in 3 different languages). I have gone ahead and used Matlab, Fortran95 and Python. I have the following questions:

* How do you test memory usage for codes such as Python, C, Fortran, Matlab?
* Is there a particular code profiling tool for Linux and/or (Intel) Mac OS X that people can recommend?

I would be interested in any comments, ideas and findings!

-Laura

Class website and the venue change

The class website is now up, and it will hopefully get better:

http://www.astro.caltech.edu/~george/aybi199/

Please note that the class venue has changed, to 153 Noyes. See you there!

Publish Post

Welcome to Ay/Bi 199 blog

Please use this blog for any and all class discussions