Random Useful References

Some references that might interest people taking this course.

- NetworkX.

A Python package for modelling graphs and networks:

http://networkx.lanl.gov/


- Graphviz.

A graph visualization tool (works with NetworkX):

http://www.graphviz.org/


- A line-by-line Python profiler

A Python profiler that gives you performance information about each line on your code, so you can attack the main bottlenecks first.

http://www.enthought.com/~rkern/cgi-bin/hgwebdir.cgi/line_profiler/summary


- "The visual display of quantitative data", a classic book by Edward Tufte.

http://www.edwardtufte.com/tufte/books_vdqi


- The NEOS server and AMPL

A server that accepts optimization jobs over the web. High-end hardware and high-quality solvers for free! Accepts jobs in the AMPL langugage.

http://www-neos.mcs.anl.gov/
http://www.ampl.com

- The Stony Brook Algorithm Repository

Search for algorithms by problem or by language. The website also provides a ranking of the algorithms (presumably which ones are "best"). For example, I needed an algorithm for matching in bipartite graphs. I just searched under "Graph problems - Polynomial type problems" -> "Matching" and found a list of them, with their ranking.

http://www.cs.sunysb.edu/~algorith/


- "Large Scale Data Analysis Challenges" (talk at Caltech next week)

Tuesday, February 24th
12:00 - 1:00pm
74 Jorgensen

*Lunch will be provided*

SPEAKER:
Dan Meiron
Fletcher Jones Professor of Applied & Computational Mathematics and
Computer Science

TITLE:
Large Scale Data Analysis Challenges

ABSTRACT:
JASON, a scientific advisory group, was asked by representatives of the
Department of Defense (DOD) and the Intelligence Community (IC) to
recommend ways in which the DOD/IC can handle present and future
sensor data in fundamentally different ways, taking into account both the
state-of-the-art, the potential for advances in areas such as data structures,
the shaping of sensor data for exploitation, as well as methodologies for
data discovery.

In this presentation we will examine the challenges associated with the
analysis of large data and in particular compare DOD/IC requirements to
those of several data intensive fields such as high energy physics and
astronomy. The conclusion is that while DOD/IC data requirements are
certainly significant, they are not unmanageable given the capabilities
of current and projected storage technology.

The key challenge will be to adequately empower DOD and IC analysts by
matching analysis needs to data delivery modalities. At a very cursory level,
we will examine some current approaches that could enable better information
fusion. We'll also propose various grand challenges that could be used to
assess and prioritize future research efforts in data assimilation and fusion.

Grading policy

Dear students,

I have clarified with the Registrar that the class can be taken either for a letter grade, or P/F. So, if you are taking it for credit, please email me:

Your name, option, and whether you want a letter grade or P/F

And, just in case you are wondering, the grading will be as follows: I will assume that the honor system is working, and that you have done all of the assignments and suggested exercises and readings. Thus everyone gets an A or a P, and now you can stop worrying about grades, and focus on learning! (In case someone believes that they do not deserve such a grade, please let me know, and I'll give you the grade you think you deserve....)

Thanks, George

Hamming numbers

Hi all,

I was wondering if we could start a discussion on the assignment of Lecture 2 (computing Hamming numbers in 3 different languages). I have gone ahead and used Matlab, Fortran95 and Python. I have the following questions:

* How do you test memory usage for codes such as Python, C, Fortran, Matlab?
* Is there a particular code profiling tool for Linux and/or (Intel) Mac OS X that people can recommend?

I would be interested in any comments, ideas and findings!

-Laura

Class website and the venue change

The class website is now up, and it will hopefully get better:

http://www.astro.caltech.edu/~george/aybi199/

Please note that the class venue has changed, to 153 Noyes. See you there!

Publish Post

Welcome to Ay/Bi 199 blog

Please use this blog for any and all class discussions