Recently, we started a new open group on Facebook for the study of cognitive analysis of programming minds. Please, do join at http://www.facebook.com/groups/ProgrammingMinds.
Prospects of Big Data in physical experimentation
A glorious chronology of physical world of discovery
The theory of Peter Higgs becomes more interesting recently due to serious experimental efforts from High Energy Physics community.
The big data generated from experimental setup and the analysis of significant evidences suggest existence of Higgs Boson as one of the strong candidates among other findings.
Though the theory of Higgs seems interesting and famous, yet I have a different perspective about it.
To draw the sequential line of events, I’m going to begin with the Newton’s master piece of dicovery.
Newton dicovered not only Gravitation but particle nature of light also. Though his theory got discorded after dicovery of light’s wave theory, it remained a milestone to the Modern Physics.
Serious discoveries about Light as electromagnetic wave and Photon particle led to the conclusion that Light has dual nature. In some of the physical phenomena, it behaves like a wave having fields and in other physical phenomena, it behaves like a particle having mass associated with it.
The strange behaviour of Light compelled people to think about the interrelation or interaction between these two types of phenomena.
The combination of these two phenomena was revealed in the Theory of ElectroDynamics, TED. This was the first time the interaction between field and particle accepted and opened a new hope of combining other interactions of nature.
Subsequent discoveries about these interactions; weak, strong, electromagnetic and gravitational interactions, led to tedious task of unification of all theories; unified theory to explain everything that is physically happening in our observable universe.
In my simplest understanding about the theory of Higgs, Higgs proposed that masses are assigned to particles in accordance with their interaction with Higgs field.
Is there exist any possibility of finding a new particle via big data?
Finally, discarding such an enormous big data and considering a fraction of it may not be a wise idea.
Mining such a big data may bring some new insight in our perception and understanding of physical phenomena and the universe, though it’s hard nut to crack. But still it’s challenging and pleasure to finding things out.
Here, I linked one of my blogs on information visualization and human cognition topics:
Until recently, the Knowledge Discovery was so successful to fulfill the need of our modern society. But recently, due to introduction of a number of different perspectives, the processes of Knowledge Discovery in Databases (KDD) changed the ways of Knowledge Discoveries.
However, the ideal processes, involved in KDD, remains the same with slight variation due to different viewpoints.
In this article, the discussion is primarily on evolution, successive development, processes and perspectives.
Once upon a time, there were great
Mathematicians, Scientists, Philosophers, Thinkers, Reformers, Artists, Musicians, Monks, Saints, Leaders, and so on. They all were quenching the thirst of the most powerful and influential factor that might change entire humanity, affect society, bring ethics, build civilization. The Knowledge to which the eternity of human being gets fulfilled with content and satisfaction. A Mathematician works out a number of mathematical analysis, modeling of problem domain to visualize and interpret the inference. A Philosopher observes the logical insights and visualize to interpret the symmetries or contradictories happenings. An Artist lays the art to visualize the sense and conveys the knowledge to Admirers. There are so many examples. The processes of Knowledge Discoveries vary in accordance with the respective contexts, scenarios, and heavily dependent on the intentional objectives.
Understanding of KDD and KDD processes is incomplete until a detailed study on the KDD processes and in turn, subprocesses within the existing processes may be identified and explained.
The most common activities that are involved in the processes of Knowledge Discoveries in Databases are listed below.
Selection of initial data —> Preprocessing of target data —> Transformation of preprocessed data —> Data Mining of transformed data —> Visualization of interesting pattern/model —> Interpretation of interesting pattern/model —> Discovery of Knowledge
It is utmost important to bear in mind that there may be additional processes or reduction in the processes as per needs, necessities and requirements of the concerned scenarios.
Overlaping of the processes may also be observed in specific situations which are though rare but might exist in the real world scenarios.
In precise words, the processes can be summed up as follow:
1. Selection: obtain the concerned data from various sources of databases
2. Preprocessing: cleanse the choosen data
3. Transformation: convert the data to common format or specified format
4. Data Mining: apply algorithms to the data for generating interesting pattern/model
5. Visualization: imagine and think about interesting pattern/model for Knowledge
6. Interpretation: evaluate and explain findings/Knowledge
The steps involved in KDD can explain further elaborative mechanism.
The first step in the Knowledge Discovery processes is collection of data from the databases.
The second step in the processes of Knowledge Discovery is refining the data, cleaning the data to be investigated, removal of all abnormalities like errors, filling the incomplete data.
The third step in the processes of KDD is to convert the data into purposeful or functional format so that the process of data mining can be initiated.
The fourth step in the process of KDD is to mine the data to evolve a useful set of data, i.e. data of desire or needed data or more precisely, generation of interesting pattern/model.
In the fifth step, the power of imagination comes into play to visualize and think about the interesting pattern/model. Visualization step/process is relatively wide topic because of vast perspectives and influential factors.
Finally, the pattern/model is interpreted meaningfully in accordance with the visualization of data. Such a visualized data is the new findings/knowledge and such a discovery is accepted as Knowledge Discovery.
More than half a million Project Managers execute about a million software projects every year all around the world, turning out software worth more than $600 billion. A considerable part of the software projects, as one of the studies, Standish Group Report shows nearly three-quarter of over all as challenged or cancelled, goes in vain due to unhandled damaging project risks associated with them.
Most of them are unidentified until too late to manage them and the results of these projects have been nothing but desperate cancellation. Henceforth, it goes without saying the need of risk identification which is a core process of risk management, the most crucial management phase of all project management cycles.
Software risk is a measure of probability for gain or loss by unacceptable result that may happen in software project, software process, or software product.
Each software system is distinctive with its own particular set of risks. There are many software risks but fewer consequences that we care to avoid. Mostly, it is hard to avoid all kinds of risks in a project. This is why we often care about a few highly impacting risks like potential cost, schedule, technical consequences in any project. These software risks could avert a software project from meeting its cost, schedule, and technical objectives. Software risk is hazardous because it can prevent software project success.
Software risk identification is the initial process in the risk assessment which is a part of risk management. Without proper risk identification, it is tough to control and manage the risk that is associated with the software project.
One of our current works is an innovative endeavor to solve the software project risk identification problem with one of Graph Mining approaches. This procedure, in turn, simplifies the software risk identification and consolidates attempts to identify software risks more vivid and robust fashion.
Further, the procedure enhances human ability and approach to identify software risks by analytic capabilities.
The software risk identification process begins with the creation of software project risk scenarios. These scenarios are at first, made by collecting essential set of data or nodes as input to the formation of the Graphs.
Further analysis of existing graphs with data visualization and interpretation leads towards concrete results. Methodology including a number of visualization techniques, for instance, Link Mining, Node Mining, ubiquotous data mining, etc., may further be choosen as per suitability of scenarios.
Flexibility and handy applicabilities are key attractive features, in addition to wider analytic investigation of software risks. Further work is due to come in coming future.
Visualization, the power of imagination is core of all sciences and technologies. Mathematics is no exception to this golden rule.
Think about ‘a set of points’, geometric figures and diagrams, it may be a simple one or be a complex one. They move, reveal new properties, intersect, combine, form entire families and change their appearance, even sometimes, reshape into unrecognizable forms.
There are a number of problems, ranging from traditional problems in which one has to find and make use of some set of points, to simple inverstigations touching important mathematical concepts and theories. All need some sort of visualization for approaching towards solution aspects. Most of the problems challange for a series of visual interpretation which can be thought of a dynamics of visualization processes.
Most of the times, it is useful to verify and resolve problems in mathematics through experiment; by drawing a diagram or graph, even a series of thoughts mapped into a series of diagrams or graphs. This experimental approach not only helps one to guess the answer and formulate a hypothesis with the aid of visualization processes, but often leads one to a mathematical proof also.
Visualization and interpretation are co-occurring integrated processes that do happen side by side for inference purposes. Reversibility between these two processes, visualization and interpretation, is quite obvious from different perspectives. For instance, visualization to interpretation is usual flow of processes but reversibility of processes, from interpretation to visualization can occur if there be a mapping of interpretation into visualization processes during learning approach as well.
The beauty of visualization can be observed everywhere and forever. It seems to be an integral part of not only non-living entities but living entities also.
Some of the remarkable discoveries of centuries are witnesses for the proof of the statement that progress in human’s history has been driven by this ultimate power of imagination.
It can be a matter of intelligence if not cumbersome or tedious, to visualize biological data for cure and secure the lives of human beings and creatures.
A golf ball could have a different perspective from the micrograph of Toxocara canis egg. But resembling such symmetry and characteristics might give new insight into the deep hidden unknown knowledge to cure in the best manner.
Most of the treatments do relay heavily on data visualization. Ciprofloxacin; an antibiotic used to treat a variety of bacterial infections including inhaled anthrax. The visualized scenario is firmly ensnared in the protein AcrB’s cavity.
Untangling complex series of events leading to the death and destruction of messenger RNA molecule is another instance where big data and data visualization contribute a lot. Two yeast cells with fluorescently labeled P-bodies.