Using Visualization to meet Big Data Challenges in Capital Markets Applications: Webcast

View and listen to the Aite Group's assertion that visualization is a top capital markets trend for 2012 since all areas of the business are demanding more insight and transparency.

  • 8c4513b5-e281-4a5d-a473-39d7f9c99a0f
  • rtmp://cp86361.edgefcs.net/ondemand/vod/videos/using-visualization-to-meet-data-challenges-in-capital-markets-applications-webcast.mp4
    • Good morning, good afternoon, good evening, ladies and gentlemen. My name is Neil McGovern; I’m senior director of Marketing for Sybase, an SAP company.
    • I focus on marketing in capital Markets and for extreme analytics and I think an interesting webcast today.
    • We have Peter Simpson from Panopticon he’s going to talk a little on visualization and give a demo and we have Jeff Wootton, he’s going to talk about some technology behind this.
    • There’s a few, little house keeping things I want to cover, at the end of the presentation, we’ll have a QandA session and at that point we’ll be asking the presenters questions.
    • To ask a question, you’ll see at the top of Microsoft office live meeting, a QandA button, if you click on that, and type a question and it will come up to our attention and we’ll address them.
    • But let me ask some of the frequently asked questions to begin with. If you want to print the slides, you’ll see the small print icon in the bottom right hand side and you can print them out like that.
    • And we’ll be sending out a link after this presentation via email that will link you to recording of this presentation. So with that, let’s get started.
    • First thing I want to do, is very very briefly just set this stage, what is Big data? So when Peter talks about how to visualize it and Jeff talks about some of the techniques and tools that you can use to harness it,
    • you understand the scope that we’re talking about. The standard classification for big data is volume, velocity and variety. But working with customers, we discover this fortunately they both begin with V as well.
    • We discovered two real world additions, value and visualization. So let’s have a look at each of these,
    • There’s going to be large increase in the volume of data, the statistics seem to be quite amazing and IBM statistics states that 90% of all the data ever captured,
    • was captured in the last two years so there’s been a big spike in the volume of data. And just by some small calculations,
    • I’ve put together I think there will be about 1.5 Zettabytes volume of data created in 2012 and that is quite a considerable amount of data. That would be,
    • you know you could take 150 trillion copies of the New York Yellow pages and stack them into end and that would be the amount of data and that would reach
    • from here to Pluto and back you know, 16 times. And I mean Pluto the Planet not Pluto the dog there in Florida. So there’s a huge amount of data being created.
    • As well as volume there is velocity and in capital markets, we’re probably more familiar with this than many other Industries because it’s far more apparent to us.
    • You can see the OPRA feed is starting to hit the grades of 45 million messages per second so in addition to large amounts of data, parts of that data are moving very very quickly.
    • The third V in the standard classification is Variety. And again variety means not the unstructured data as well as structured data in this case. In capital markets,
    • we’ve actually been a little bit out-front with handling the variety of data. We’ve been using unstructured data in algorithms and you can see some of the figures here from IT group
    • about the use of unstructured data and trading strategies and the types of unstructured data.
    • I presented with David Leanweaver at a conference last month and he talked a lot about how news and sentiment analysis can be very effective in trading environment
    • and we demonstrated at an Industry that this is a valuable source all of information.
    • But there’s a couple of other V’s, there’s the Value, I talked about the value for the trading strategies.
    • Finding the nuggets of data in this ever increasing ocean of data is getting nuggets of value in this ever increasing volume of data is getting more and more complex
    • which is so much more information and making sense of it is important. For good handling unstructured data, that’s something we’ve been doing for many years.
    • We’re just learning as an Industry how to really get value out of unstructured data and a lot of mystery around big data, I think falls into this Value area.
    • To get that value, people are discovering that you need visualization. And this allows an interaction between the analytic tools and human beings.
    • In some cases tools are very good at seeing patterns in other cases humans can see patterns that tools struggle with.
    • So, this combination of being able to bring together the strength of the analytic tools with the strength of the visualization is what this webcast today is about.
    • The experts in visualization are Panopticon and we have Sybase SAP, have some very strong tools which Jeff Wooten is going to talk about in a minute to help with getting the value out of the data.
    • But first, let’s have a look at visualization. And I would like to hand over to Peter Simpson, in a second. But first, I want to ask a question. I’d like to see what the audience thinks.
    • ‘What business applications do you see that will benefit most from implementing Big Data technologies’? So if you could just choose your answer here. You can see we have four different solutions.
    • So will Big Data be good making real time trading decisions and afterwards we’ll bring all that data into trading decision window or will you use it to look up to create better strategies.
    • Is this more of a risk management opportunity or is this something that’s required for compliance. So the results are coming in, and it looks like,
    • Norma if we can just stop the polling and now you can see that about 50% of you think that this is going to be value during the risk for risk management.
    • A significant portion also believes there is a trading aspect to this as well. So thank you for that, and let’s move on to our demo.
    • And today our demo is going to be conducted by Peter Simpson. Peter Simpson is Senior Vice President of research and development for Panopticon software.
    • Peter is responsible for all development, pre-sale support, consulting activities. Peter helped content,
    • product management and implementation roles instant information before joining Panopticon in 2007 and he did many senior positions including HPC global,
    • global distribution manager for global Research and e-Commerce strategies.
    • Peter has a Master’s of Science degree in information systems and a bachelor of science in Physics which is based in scientific technology.
    • With that, I’d like to obviously hand over to obviously very well qualified Peter Simpson. Thank you Peter.
    • Thank you Neil, ok just a very quick background in Panopticon focusing demo. Panopticon is a 12 years old company and we’ve been a partner of Sybase for a number of years now globally.
    • We focused on real time visual analysis, historical time frame analysis, connectivity to both tic databases columnar databases,
    • such as those provided by Sybase and real time systems whether CEP or message buses. And as well as providing packaged tools, that allow people to embed what we produce into their own applications.
    • Now about the product set, really comes in three flavors. The EX packaged application which we’re going to see today, here we produce a visual analytics,
    • what looks like a level of SDK from Panopticon developer and a rapid development kit where people can write their UI’s in the desktop production that goes in their packaged application.
    • Today we’re going to focus on EX.
    • So demonstration and I’ll share my screen.
    • Ok I hope you’re able to see my screen. And I’ll just swap to ESP, so today we’re going to be looking at Sybase ESP and Panopticon EX.
    • First we’ll start on ESP, this is my calculation and state for aggregation hub, it’s providing the heavy listing I need. Now, I subscribed to streams for windows,
    • as you can see on my right hand side even in their entirety or filtered use of the data and by subscription to any offline streaming.
    • That is data set. Now you see a portion of my bigger model here, visually displayed in a directive graph and this is in the ESP studio.
    • Now this model allows me to consume trades, historical data, current market pricing whether that’s actual in search and calculate position metrics against minutes and against previous different state closes.
    • Now some of the classes and some systems are more liquid than others and then consequently the data feed is higher,
    • the objective in what I’m trying to see and send to my positions across my example bank, it’s the most up to date picture of my position as possible.
    • So if something’s trading once every day, I don’t need real-time data. When I’m looking at currency trades which are trading kind of intra seconds then I want to have that data as real-time as possible.
    • Now, in my demonstration each trade has 3 primary dimensions, the desk book, the underlying instrument and counterparty.
    • These are locked to 3 distinct hierarchy, same in the case of desktop to offices, asset classes, countries, regions and globally and… Instruments and counterparties.
    • Now ESP is running on server hardware and scales based on the required data percent for your model and Jeff will cover that later.
    • So then all that put together here I’m in the Panopticon EX here I’m in the EX Designer and we create and display interactive visual analysis dashboards,
    • using this tool which can either be simple or complex depending on the end users requirements. At the moment I’m in the desktop based designer as showing a pre created work book,
    • now this could be displayed elsewhere in the desktop tool or our web browser or embedded into an existing trading or risk application through our development tool capability.
    • Now no matter the deployment mechanism Panopticon EX is fundamentally desktop based, in that the user interface created in front of,
    • front office personnel typically the risk, research del for trading, we will enter the presentation mode, right on top.
    • I only used The design elements from the page and I could start subscribing to my ESP model. Now here I see a top level traditional tabular view of my 3 hierarchies, business and line and counterparty.
    • I’m focusing on first figures, market to market, delta and gamma into VaR calculations, 1 day VaR and 10 day VaR both calculated at 95% confidence levels.
    • I’m particularly interested in the current values, the change on the day both in absolute and percentage terms. As I’m viewing the tops of the hierarchy,
    • I see top level massive or aggregated data and in this case, the market to market facility, … and their additives, in the case of VaR and I can’t approximate for them to be additive,
    • so I need to pull down the aggregate values from ESP as well as release values. I’ve only been able to calculate them on the desktop, given we want to minimize what we do to the trade machines,
    • we’ve really wanted to impact the trading applications, now here I see the value, and it continues to update if I get any data in, followed by a change since yesterday,
    • each figure is colored based on the 10 percent change in the day, bright red or bright blue. Blue would be 10% and red would be white and they change, I’m using red for bad and blue for good.
    • When the market to market increases we show blue and when it decreases, we show red as we have an effective which really is effective penal measure.
    • In the case of sensitivity and VaR number’s we’re swapping with scale, red means for example increase in risk, the one’s we’ve got in our group position and the blue would be decrease,
    • in both cases red bad, blue good. By the tabular, we use work well when we see small number of items but they fall off as the volumes increase.
    • If I change the bottom hierarchy we’ll be looking at counterparty types, I look at all my counterparties, I see numbers and I’m intimidated by the amount of employment data,
    • here I got a small data set but still I’m having to use the scroll bar to introduce and the color shading helps but I can only see the rows on the screen and the changing data makes this,
    • we’re looking at real time streaming data in fact is so if the volume of data increases and I get into the current data volumes I need an attempted approach. This is where visual analysis adds its primary value.
    • And we’ll connect into this slowly, so here I see the same data visually, we started by allocating a box to each counterparty,
    • we changed the color of its box to be the market to market change in the day, again and down 10% up 10% and see the visual icons … before heat map.
    • Now in this case we have around 60 to 70 counterparties so this approach would work well if you have hundreds or thousands or tens of thousands.
    • And this is practical because we don’t have to have each counterparty of equal importance, of equal size,
    • we can just change the size to be in this case market to market value important in the items to us and those of the biggest position sizes would be on top left
    • and those of the smallest position sizes would be and least important to us appear bottom right. And that would scale pretty much independent to the data volume,
    • now we’re going to add a hierarchy and I’m going to add counterparty a scale on counterparty and we can see average positions by counterparty rolling up to the down style and see where we’re doing badly,
    • where we’re doing okay. When our counterparties have increased risk, with decreased risk on that position size and we can see if we look again at 40 different counterparties are….
    • Now I can take another hierarchy with interest so let’s go, an asset class, an underlying sector and I can see the breakdown of positions by asset class and by underlying sector,
    • if I want to change the view, so let’s say I want to underlying sector and then asset class I’ll just re-arrange the hierarchy and when we rejig, we re-build our aggregate when we say we need value.
    • I have a big errand commodity followed by currency and then we see each sector divided into whether it’s equity fixed income or mutual fund.
    • Now if I want to see the counterparty exposure within an asset class, I just update hierarchy again. Add, select counterparty and now we see each of the individual counterparties.
    • Now the key here is to choose the appropriate level of detail by each rapid understanding.
    • So now we see an incoming introduction both how you build up visuals and how you look at data graphically. Let’s look into the details of the hierarchy.
    • Click on a hierarchy tab, start subscribing to the data again now we can see desks, scrolling up to asset classes, offices, countries and regions.
    • So currently I have a top level region at the top and all the way down and I can change that if I want to look across the asset classes by region, just re-arrange the hierarchy again.
    • Now that rejigs. Now, I get the same problem I get with counterparties when the clips give me a rows of data, if I minimize table, maximize tree map, now I can see my hierarchy again.
    • I can see the top level data, the aggregate values all the way to here and down to the recent values again. Increase the volume of data
    • and you can still be able to pick out the important items among those who have the biggest variance plus the overall aggregate positions.
    • Now let's look at a more finished dashboard. Here I see the top level of information displayed in the biography on the left hand side, I can see the asset classes by Country,
    • the height of the bar is market to market color again it's the change on the day, I can investigate the hierarchical performance through the tree map at bottom
    • and I can investigate coalitions using the Delta Chg at the top.
    • I'll be going on to make a way for goal numbers just so we have too many numbers now in fact focus on the overall position, start focusing on cost string, the data trends and outlines.
    • Now the first criteria on the right hand side, has your purpose firstly, if I already have an area of interest, I can easily focus on this area, let's go asset class,
    • de select everything and select equities , now I'm seeing just my equities to view the world secondly, for numeric measures I can see the distribution profiles.
    • In this case I can see the current data figures plus yesterday's closes, plus the absolute changes on the day and I quickly screen the outline I knew my hand wasn't focusing on the tale of the distribution.
    • Now this hierarchy is looking at my business context, regions, Country, office, asset class desk.
    • If I go to this tab, its symbol visual we start describing again and I can visualize positions in the underlying instruments.
    • Now here the volume of underlines is much greater as you can see from the counterparts, every underline I have in my business again the same principle applies,
    • summary top level data in the biography on the left in this case my asset class and by domicile, again it's the domicile of the underlying rather than domicile of the counterparty,
    • we picked variables that make sense for you to display. Plus visuals such as the tree map and…
    • Which we deal with by looking at large volumes of data but focusing on the overall picture plus outlay trends and clustering of data
    • and finally I can do this again in my hierarchy which is my counterparty view and here you'll see each counterparty.
    • In each of these three cases, as I move between tabs mixing dashboards I'm prescribing seamlessly to the appropriate aggregated streams and position metrics from my underlying ESP model
    • and when I select a counterparty of interest, this one here, I drill I decide instead of looking at counterparty level, aggregated data, I go into this tab dynamically,
    • and I dynamically subscribe to a new data set. In this case I'm subscribing to every position owned by Deka investments rather than just aggregates.
    • Now, here this bead dynamically is creating a stream for me to subscribe again and in Panopticon search we treat both data sets both each of these individual positions and then we are going desk.
    • Again as before I can quickly view the summary level metrics to my left, through my bar graphs or focus on investigating outlines through tree maps catalogs or other visuals we provide.
    • Now here I have two sets which I'm looking through my asset class and business hierarchy where I have exposure to Deka with each book and office running up
    • and where I had exposure's by underlined whether they had big position commodities and assets and equities and fixed income one can drill down to grow the joins from it again.
    • And look at market to market exposures sensitivities and risk measures. Now if you compare this approach of looking at the details drilling in and we can keep drilling in if necessarily,
    • we can see how quickly we can get to opportunities and issues that we want to investigate and often here, I could flex the position and quickly drill into another screen that would look for trades
    • that make a past position plus the timing of those trades. Now the technical message is exactly the same Now, if you compare that,
    • back to the tabular view of the world we can see how we can much more effectively plot risk opportunities and act on them much more quickly
    • because we're going from looking at 5-10 rows of data to looking at tons of thousands of rows of data very quickly.
    • We are moving away from looking at old numbers that are as you expect them and focusing on the top level outline trends clustering of data.
    • Ok come of presentation mode, we stop streaming, just a little behind the scenes, if I click a visual as you can see here, I can see the associated data layer behind it,
    • I just edit this, I see the underlying data structure and I can see the calculations I've added on my side,
    • I click settings I can see the underlying schema of the stream I'm subscribing to I may display them and if I expand this data plug in settings, you can see here I'm connecting to my ESP further,
    • on particular host imports, I'm specifying the model I want to connect to or the application name I'm allowed the streams that I want to subscribe to. Now in all of these cases,
    • I'm sitting on the front end of the trade with a desktop that I'm subscribing to this data in this kind of big filtered aggregation hub from calculation hub and that's ESP. so finished the demo.
    • Thank you very much Peter. So let's quickly ask another question of the audience and so the first question, 50% of the people said that risk would be important to them for a big data perspective.
    • Let's look at the velocity side of the equation as well, based on this, ‘how soon do you want to see your positions and exposure'? As soon as possible, hourly, 3 times per day or intra-day or daily?
    • Ok Norma, thank you. Ok you can see the vast majority of the people here realize the value of real time information and being able to handle data like this in a fast format for risk exposure.
    • Let's take a quick look behind the scenes and have a look at some of the underlying infrastructure that can manage the Big Data and set it up
    • so that Panopticon can visualize let's look at some of the tools that underneath here that can pull the value out of the data and to do that, we're going to have Jeff Wootton join us.
    • Jeff Wootton has over 20 years of experience in technology for market data,
    • trading and risk in the Capital markets and currently leads product management team for complex event processing at Sybase, SAP so Jeff?
    • Thank you Neil, so one of the things that I'm sure you noticed in looking at Panopticon EX and one of the most notable features that sets Panopticon EX apart from any other data visualization tool,
    • in the market is that those dashboards, those graphics you were looking at were live you saw them updating as you watched them.
    • They are constantly changing to reflect the current set of information that you're trying to understand.
    • So, to do that Panopticon needs a source for that data that is constantly updating as new data arrives and that source, that we are showing today is the Sybase event stream processor also knows as ESP.
    • Sybase event stream processor is a high performance complex event processor or CEP engine that continuously processes streams of event data as fast as that data arrives to extract insight
    • that can be delivered to users in the form of dashboards such as you just saw with Panopticon and the word can be delivered to the downstream applications that are going to act on that information.
    • The key is that ESP is continuously processing all the incoming events,
    • in this case the key event streams for this demo are trades that are affecting positions and market prices that are being used to value those positions.
    • The data model set up in the ESP to determine, how to process those incoming events? How to combine information across streams, how to aggregate that information,
    • filter that information etc. to get it into a useful form so you can deliver that to end users. The ESP engine can be used to watch for things to happen and alert users when something has happened.
    • It can be used to capture information into a database such as Sybase RAP, Sybase IQ or SAP HANA for further on demand historical analysis of what's been happening.
    • Fundamentally though, for those of you who aren't familiar with CEP or Sybase ESP in particular,
    • you can think of it as the type of data analysis that might traditionally be done using a database with static data,
    • where you collect all the data in a database and then query that database to analyze the data and in fact with traditional visualization tools,
    • that's what you would do, is with your visualization tool you would query the database and then grasp the data. With ESP you can do that same sort of analysis but on a continuous bases,
    • it's sort of like turn it upside down and instead of first collecting the data and then querying it with ESP you define your queries in advance,
    • we call them continuous queries and then they sit there as live queries and as the data flows in, it passes through those queries to continuously update the model.
    • Now one of the things that's unique about the Sybase ESP architecture and that really sets it apart from other CEP engines in the market is that it has a notion of windows
    • that have state and for those of you who are familiar with databases, you can think of them as materialized views in a database but the difference is,
    • they're not only in memory but they are part of that dataflow model so they continuously update and in fact, inserts,
    • updates and deletes can be applied to those windows so that they reflect not only a collection of events but the current state of a set of information so for example,
    • the current net positions, in the case of my real positions and what's more is on the output side applications can subscribe to those windows and get the full state of the window with all the changes to the window.
    • So it's very different than a traditional approach to CEP that just considers each event as an independent event,
    • tends to approach events in kind of a stateless transient nature for your watching for things to happen.
    • You can certainly do that with ESP but it's also very powerful in its ability to aggragate events and maintain sets of information in memory.
    • You saw a little glimpse of the ESP studio that Peter showed you there. One thing that's worth mentioning with the ESP studio, the studio is an eclipse based studio that's where you set up your data model,
    • it's where you define the continuous queries that will be applied to streams of events to produce that inside, that summary information, those alerts that you're looking for.
    • The ESP studio has 2 different editors, there's a visual editor as well as a textual editor. Basically this just lets it appeal to a wider range of users.
    • Lot of people who aren't programmers are much more comfortable working in that visual editor where they don't have to learn the underlying language,
    • they can drop queries onto the diagram, direct the flow through those queries, for some programmers they find it more efficient to work in a text editor.
    • That visual editor produces an underlying textual model that can be edited directly and users can switch back and forth between those, you want to get a closer look at how models are built in ESP just let us know.
    • And finally, just a comment that we are looking today, we showed you one particular demo around real time position and risk monitoring,
    • that's just one example of the ways that Sybase ESP and in fact Panopticon EX are being used in the capital markets. It's really been applied to a variety of use cases both pre trade and post trade.
    • On the pre trade side we are seeing a lot of use around market analysis whether that's simple market data enrichment or analyzing markets looking at consolidated prices on order books across different markets,
    • analyzing market depths and volatility being used for automated security pricing in different markets,
    • quite a bit going on lately with trade monitoring applications and then there's the post trade application such as you saw today.
    • So just to give you a flavor of what the Sybase event stream processor is doing behind the scenes of these dashboards and how it's being used in the capital markets.
    • So with that, I'll hand it back to Neil and I think we're ready to take any questions.
    • Thank you very much Jeff, yeah so the moment, there's a QandA button at the top of your screen, we have some questions in the queue which we'll start off with,
    • if you have a question, just type it in and we'll address it.
    • First question Peter, from Shankar.
    • VaR is not additive as we go from individual assets to the overall portfolio. So how do we address this as the tool by default we'll sum or use weighted averages,
    • I think the question is are the sophisticated mathematical analysis techniques you can apply at a tool level to handle things like manipulating VaR to get a clearer picture.
    • ok from a Panopticon perspective, we're not calculating VaR with plain VaR and as it's not additive we develop putting down belief sets of data
    • so the bottom of the hierarchy needs to look down aggregate value as well and if you want multiple different hierarchies and people to manage and change the hierarchy,
    • we have to pull down additional aggregates with those different combinations and we subscribe to ESP and pull down those risk numbers.
    • Thanks Peter. So I think this question is more directed towards Jeff.
    • Jeff, how do you scale with data volumes in CEP so that you can handle the big data like volumes that we talked about at the beginning of the presentation?
    • Sure, so the Sybase ESP engine is designed for high throughput and scalability from the outset. So to start with,
    • a single ESP server running on say 4 cores can easily process hundreds of thousands of incoming events per second.
    • Now as you saw in some of the examples, maybe not reaching to Pluto but certainly things like the feeds from Opera and other exchanges,
    • there are plenty of cases where we see we need to go in excess of hundreds of thousands of messages per second,
    • we've even seen use cases in excess of a million messages per second and for that, first of all an ESP server scales across any number of cores,
    • it's a multi-threaded server so fundamentally the more hardware you make available to it, the more performance you can get out of a single server,
    • you can also start to run parallel processing paths to even further expand across not just multiple cores but even multiple servers.
    • So you can break that data stream across different parallel instances, across different machines, and then expanding up to a full cluster of ESP servers,
    • so very scalable to get, to handle whatever data, whatever size data flow you want to throw at it, really.
    • Ok so Christina just asked a question about your response which was, does this require a lot of memory for ESP and does that impact the scalability?
    • Yeah good question, the memory requirements are really driven by the data model.
    • So when I say data model I mean what are those continuous queries that you set up and I also talked about the ESP ability to hold windows and state of data and yes that's all held in memory.
    • So the amount of memory required is really driven by how much data you are going to hold in memory.
    • So there is some data models that require very little memory because the data is coming through very quickly but the amount of data being held is relatively small,
    • we have other models, we have some users holding days' worth of transactions in memory data model and as you can imagine those have some fairly hefty memory requirements.
    • Right you know memory is getting a lot less expensive, there hopefully should be less of an impact.
    • There are a couple of questions more for you Jeff, I want to go through one for Peter here, Peter this comes from Tony.
    • On the capabilities of hierarchy and business context for visualization,
    • is the hierarchy defined in the visualization layer dynamically or does it have to be pre agreed in one or more entities in Sybase data analytics layer?
    • ok the hierarchy is a set of columns, it's a set of text columns in the underlying data source and that could be coming from ESP or it could be coming from another database where we're joining in memory on the Panopticon side
    • it depends how you want to go. As, we have a set of columns then we can just drag and drop and arrange those columns to produce the hierarchy that you want to see
    • so as we saw earlier in the demonstration, I had global region, country office, asset class desk and when I dragged the asset class to the left and it rejigged and,
    • in the case of the additive numbers, I like market to market in sensitivity, they just rolled up and in this case of non-additives like VAR,
    • I have made sure that ESP was providing me loads of aggregates for that new hierarchy as well.
    • So, actually this is a question that's come across, so I'm going to ask a concept question of about 3 or 4 that are in the queue,
    • I think there's an interest in knowing where the calculations happen? I think that some calculations are very complex and people have underlying engines some calculations like VVar
    • which Jeff talked about where you have a window of information that might be a few minutes or a day where you calculate your V VaR or TVaR can be done in ESP
    • and then lastly there are some complex engines that calculate things like maybe VAR or other calculations.
    • Jeff can you talk very very briefly about that, is that three layer structure that I just said indicative of what happens in the field?
    • Yeah that's a good description Neil because it really is the case.
    • There is, calculations can happen several places and typically they do so some calculations are being done in the ESP engine in real time and this example uses quite a bit of aggregation in the ESP engine,
    • so you mentioned V WAP for example, a real time V WAP is a very straight forward real time calculation that ESP can do very quickly,
    • some calculations are being done in Panopticon and those are the ones that let that dashboard be interactive so that the user can look at the data in different ways and then there's some calculations,
    • typically things like VaR calculations or some of the modeling, they really don't lend themselves to real time,
    • either they need to look at large sets of historical data and or they require an iterative calculation such as a money current simulation,
    • those sort of things and those aren't realistically going to be done in real time.
    • Now the results of those may still feed into the real time model but those are going to be done offline and the last thing worth noting is that ESP can also use proprietary external functions
    • so there is a user defined function interface and the ESP engine in real-time as part of the event processing engine can invoke external functions
    • that do things like maybe a proprietary pricing algorithm or something like that, so the calculations can get done in several ways but combined here to show the user the end results.
    • Ok cool, so I'm going to put you in this spot Jeff and Peter to an extent because 3 or 4 of the questions in the queue talk about the footprint and the cost, the pricing structure and so on,
    • so maybe if you can talk very briefly about the cost of each individual tool, Jeff you go first and then Peter ,
    • Jeff as you finish talking if you could talk a little bit about the cost of the larger footprints of memory the disc and so on,
    • I understand that this is asking how long a string is but I think the audience wants to get some feel as to the sort of budgetary and pack to all a solution like this.
    • Sure so Sybase ESP it is priced per core. And it's a standalone product in that,
    • you can buy yourself so you don't have to buy it in conjunction with other Sybase product now of course it integrates with other products like I mentioned, Sybase IQ, Sybase RAP
    • but you can buy it by itself, it's license per core. We recommend a minimum of a 2 core machine but it runs on basic commodity hardware, Intel architecture, Linux operating system.
    • As I mentioned the amount of memory is really dependent on what you're doing with it so bottom line it doesn't require high end specialized hardware.
    • In fact I'm pretty confident Peter was just showing it to you on his laptop probably and then the per core pricing is really again just driven by the size of the data model you're running,
    • how much data you're pumping through it and the complexity of the calculations.
    • So in the scale of like a car, a house or retirement, we're talking like the car level not the house or retirement?
    • Oh yeah
    • Ok I think that's the scale, Peter could you talk very briefly about the license structure for Panopticon?
    • Ok Panopticon has a 3 product set where we realistically they work in a similar way there's a end user license and there's a designer or developer license.
    • So if typically we see a small number of developers and designers and a large number of end users that we want to interact with the data
    • so that they can slowly need to know how and where the data comes from and how it's pulled together. There's ... delivering by the web, there's also a kind of Web server deployment license as well.
    • And just add on to Jeff's point, previously in terms of the scaling, where sitting on the desktop, what we do intends to be,
    • we're looking at aggregated or filtered data sets that we all would have to have lots of memory.
    • Footprints in the demo that I was showing I was pumping through 200,000 messages per second through ESP on my little DELL laptop.
    • Ok so on the scale of car, house or retirement; you're actually talking about an expensive dinner in the steak house in New York? Is that where we are?
    • Where the components of the clusters, ESP or in equivalence of Sybase technology.
    • Ok so this is technology within range as you started obviously as your foot printing commands where people were using visualization, the number of models you're using to crunch a larger footprint, you need for ESP you can scale,
    • but this is a technology that has a very large upfront. Let's see what else we have, we covered the licensing, and we covered the questions from the infrastructure.
    • So I think we're almost wrapping up here. Is there any last point Peter you want to make?
    • From the infrastructure on outside, we're running on someone's desktop one with windows application, and the service, we're running on commodity hardware.
    • So we're not buying anything in specific and I'm sure Jeff can add to that.
    • Jeff anything last you want to say to the audience before I wrap up?
    • No I think we've covered it.
    • Ok so well thank you very much! Thank you Peter, from Panopticon and one of our most favorite partners at Sybase, SAP and marvelous technology
    • if you get a chance to see it live and play along with it, you'll realize the power of this technology and the technology is driven by the ability to present the data to it in a format that it can visualize
    • and Jeff talked a little bit about how Sybase's event stream processor or ESP can handle these large volumes of data and this a necessity,
    • the value and getting the value and visualizing are the two V's that aren't talked about enough when we see Big Data presentations, when we see our customers addressing big data issues.
    • So with that, I'd like to thank everybody very much and wrap up Norma?
  • 4x3
  • Play Again?
  • 512
  • 288
  • video Using Visualization to meet Big Data Challenges in Capital Markets Applications: Webcast

Closing this window will discard any information entered and return you back to the main page