Safety First: Stochastic Decision Problems

Vladimir Norkin of the Glushkov Institute of Cybertronics in Kiev gave a very interesting (and very equation based) seminar on mixed integer approaches in stochastic optimisation on Friday afternoon. Norkin has been a visitor in IME for the last week. Titled "Safety First Class of Stochastic Decision Problems: Mixed Integer Approach" he explored examples of portfolios with critical thresholds.

Portfolio selection is a relatively new area for me, something I have learnt since arriving at IIASA. Its interesting to see how the idea is used in a variety of sectors and the philosophy of risk minimisation by selecting a diverse portfolio. Safety first means the minimisation of the probability of bad events.

Norkin started the seminar with a quote from Ulrich Beck (1986) that "society starts to realise its exposure to changes including those that appear due to its own development..." and then moved on to AD Roy's (1952) paper on "Safety first and the holding of assets". Its interesting to note that while mathematical techniques and computational capabilities have increased since then, the idea and fundamental theory still holds true.

Norkin also discussed techniques associated with a finite scenario set that relates directly to my YSSP project. He intimated that all finite scenario set probability optimisation problems can be reduced to a mixed Boolean programming problem building on Korbut and Finkelstein (1969).

Is the MEAN meaningless?

After a rather heated debate last night about climate models, betting on the model runs and trying to analyse the data from these massive collections of model runs (ensembles) I have a few thought provoking questions for people:

Is the mean meaningless or useful?
What would you do if the mean is consistently wrong?
Would you bet on the mean of an ensemble?
How do you cater for the variability shown by the forecasts, especially if they encompass extreme that can feasibly happen?
Do you feel its responsible to give a policy-maker or decision-maker the mean without a proper feeling for the envelope of possibilities? Could policy-makers design policies that used that variability? Would these be more robust than the current solutions?

There Be Science

As some of you know I have been fighting with my research topic for the last two weeks with respect to its validity, relevance and feasibility in the context of southern Africa. I'm still not convinced and hope the work I do here is not in fact 3 months of 'wasted' work as I potentially go back to the drawing board when I get back to CSAG.

If you walk into my office at the moment you'll notice a blackboard. When I arrived it was filled with financial equations and I laughed thinking I'd never need to start writing equations...famous last words. The board is now emptied of finance equations and now filled with DSS equations and (its an old record photo) goal functions with safety constraints for STO. Are you filled with as much enthusiasm as I am...

IME: Modeling Primer

Yesterday afternoon saw the second seminar of the day, 'Modeling Primer' by the IME group. This lecture delved into the world of rational decision making: making the right decision at the right time. The right decision takes experience, intuition and knowledge (facts and rules).

It is a myth that it is easy to find a solution to a complex problem, for simple cases the decision is correct but you are taking a chance for complex problems.

A model is an adequate (for a purpose) representation of a part of (interdisciplinary) knowledge. I reiterate a quote from the beginning of June: All models are wrong but some are useful. They provide a tool to HELP find the best decision. BEST is merely a suggestion of decisions for reaching certain goals which can be done through analysis of trade-offs between conflicting goals or an evaluation of the consequences of various decisions.

A model requires consistency, transparency, replicability and documentation. Something to remember is that RARE events do happen, the low probability mean it won't happen.

Some parting words of wisdom from A. Einstein - "Everything should be made as simple as possible, but not one bit simpler." Words to live by.

IME: Multiple Criteria Analysis

Marek Makowski gave a seminar today on Multiple Criteria Analysis for the Integrated Modeling Environment (IME) group.

Integrated model analysis helps one analyse a manageable number of solutions having different trade-offs between (conflicting) criteria. There are two different types available. Descriptive (simulations) which is alternative-focused. The input is the decisions and the outcomes the goals. The other is Prescriptive (optimisation) is goal-orientated. The inputs are the goals (preferences) and the outcomes are the decisions. Where there are goals, one needs multiple criteria analysis.

IME IIASA developed a Multi-criteria Analysis tool for an energy project a few years ago. The tool is fairly easy to use and available to the public (along with some introductory and advanced papers). Our next session is a hands on workshop for using the tool.

In other news I skipped Adaptive Dynamics this morning, the PHD Comic below seems rather apt, the lecture venue is literally just down the hall.

Decision Analysis: How certain are you?

Its amazing when one wonders through the library at IIASA. Its like stepping back in time to an age of paper journals and stacks. You even have to fill out paper loan forms. The librarian is so helpful and put together a unique reading list to deal with my topics and methodologies. It is the kind of personal service that makes you want to go back time and again. Imagine UCT library with a personalised service.

My current reading list is full of Decision Analysis and Robust Solutions. The founders of decision analysis were all affiliated to IIASA in some form, Ronald Howard being one of them. Its quite scary to grasp just how far their network of influence goes.

Decision Analysis has been around since the 60's, so why am I reading about it now? Bayesian networks, influence diagrams, decision networks are all founded on this concise technique that captures the most incredible amount of data, both subjective and objective. My pile of papers is growing...

Structured Modeling Technology

"Structured Modeling Technology (SMT) is based on the paradigm of separating the model specification from its data. Model specification contains the semantic descriptions of the model entities (variables or parameters) and the relationships between them. Based on the model specification it is possible to generate the data structures in the data-warehouse (DW) to store the model data".

SMT was designed to support model-based analysis of complex decision problems. There are many advantages to divide, conquer and integrate (see computer science theory) such as one data repository, reuse of specifications etc.

The Integrated Modeling Environment (IME) group at IIASA developed SMT in 2004. I am hoping to learn how to use the same fundamental modeling techniques (divide, conquer, integrate, document) while developing my thesis methodology.

2% Solution

Today we had a talk by Crawford Stanley (Buzz) Holling. Holling is an ex-director of IIASA and works in Ecological and Evolutionary Sciences and works with the Resilience Alliance. His talk was officially labeled "Broken Society, Broken Ecosystems, and Broken Planet". His talk though was primarily on the 2% solution. 98% of the time, the linear solution works. The data makes sense, but what about the 2%? Who cares? Well apparently all Systems Analysts should, this 2% falls within the realm of the exciting, the challenging and the place that allows us design new methodologies to tackle non-linear problems.

Another interesting quote came from a fellow YSSPer, "All models are bad, some are just more useful than others". Good words to live by at the moment.

Visiting SAWS

Simon Mason from IRI is visiting South Africa for a project with the South Africa Weather Bureau (SAWS). I had the opportunity to go to Pretoria to ask a few questions to the expert himself. Below I have listed a few things I learnt on my trip:

There are 3 attributes that one must consider in verification of forecasts:
• Resolution: Conditioning on the forecast. If you forecast something, does it happen? A forecast is useful if you have some resolution. Difficult to measure (need lots of data). Good for regions not grid boxes.
• Discrimination: Conditioning on outcomes. If it rains, did we forecast it. Needs less data. Can get map
• Reliability: Measures confidence and bias. Not useful on its own. Need to use (Resolution ∪ Discrimination) ∩ Reliability

Defining a region of study:
It is worth starting at 20 degrees latitude and longitude on each side of area interested and then narrow down by looking at diagnostic maps from GCM to downscaled. Easy to do with CCA. If looking at geo-potential height as an indicator, then the region may need to be bigger to include larger scale synoptic conditions.

Some extra literature for my spare time:
[1] R. Hagedorn and L. A. Smith. Communicating the value of probabilistic forecasts with weather roulette. Meteorological Applications, 2008.
[2] S. J. Mason. Recommended procedures for the verification of operational seasonal climate forecasts. 2010.
[3] A. H. Murphy and D. S. Wilks. A case study of the use of statistical models in forecast veri- fication: Precipitation probability forecasts. Weather and Forecasting, 13:795–810, September 1998.
[4] M. S. Roulston and L. A. Smith. Evaluating probabilistic forecasts using information theory. Monthly Weather Review, 130:1653–1660, June 2002.
[5] M. K. Tippett and A. G. Barnston. Skill of multimodel enso probability forecasts. Monthly Weather Review, 136:3933–3946, 2008.

Probabilistic Network Library

The Intel Probabilistic Network Library was a project originally developed by Kevin Murphy and then translated from Matlab into C++ by the Information Technologies Laboratory. This library has not really been supported since 2004 but has an interesting high level wrapper library to its underlying functionality. For those looking to use a Bayesian network library, maybe look at SMILE.

Met Office

I boarded a train at 8.30 this morning for Exeter. What's in Exeter? The Met Office. The SAWS equivalent with more money. We got a taxi to the office, nice change to walking and trying to catch buses. The taxis wasn't allowed into the grounds until he identified the passengers. The Met Office is very tight with security. We were escorted to the bathroom!

The visit itself was mind-boggling. The place is huge. Has its own gym and cafeteria. Bruce gave a talk about ICA and stakeholder needs. Between all this I met a variety of researchers, all of which explained their research. I now have a massive task of trying to remember the content and go back to look it up when I have more time. Amongst this is: Land use change, snowpacks, perturbed physics, downscaling, ICA, DFID, fires in RCM, albedo from burning, vegetation feedbacks, splines, SOMs hierachies.