This project will investigate the use of large-scale social media data capture to identify, construct, and visualize risk narratives: conversations formed from explicitly or implicitly related social media posts over time. We will build narratives based on an ����������������anchor topic��������������� of interest, estimate sentiment on the text within the narrative, then present the social media posts and related narratives using an interactive, web-based text visualization system that represents posts, sentiment, narratives, and other text properties using a suite of text visualization techniques. Our overall goal is to assess the strengths, limitations, and capabilities of our system, and to determine how to scale the text analytics and visualization approaches as the social media database grows.

This proposal is for the development and evaluation of CAPTIVE, a Cube with Augmented Physical Tools, to support exploration of three-dimensional information. The design of CAPTIVE is founded on the concept of tool use, in which physical objects (tools) are used to modify the properties or presentation of target objects. CAPTIVE integrates findings across a wide range of areas in human-computer interaction and visualization, from bimanual and tangible user interfaces to augmented reality. CAPTIVE is configured as a desktop augmented reality/fishtank virtual reality system [120], with a stereo- scopic display, a haptic pointing device, and a user-facing camera. In one hand the user holds a wireframe cube that contains virtual objects, in the other the pointing device, augmented to reflect its function as a tool: a probe probes for pointing at, choosing, and moving objects; a magnifying or semantic lens for filter- ing, recoding, and elaborating information; a cutting plane that shows slices or projection views. CAPTIVE supports visualization with more fluid and natural interaction techniques, improving the ability of users to explore and understand 3D information.

DO7 SCADA

Our goal in this project is to extend and improve an initial investigation of ways to visualize deep neural nets (DNNs) that was conducted in the last half of our 2015������������������2016 SAS research project. Based on a prototype developed during last year������������������s SAS project, we will: (1) develop methods to visualization convolution deep neural networks (CNNs) and recurrent deep neural networks (RNNs), and (2) design methods to query structural elements with a DNN to better understand its ����������������purpose��������������� in the overall context of how the neural network is performing its assigned task. Included in these two goals will be a study of how to scale the visualization to larger DNNs, and an investigation of whether achieving our two goals will assist designers in optimizing the DNNs they implement.

For this effort, we will extend the question answering system currently in place at the LAS to augment it with sentiment, narrative, and visualization capabilities. In particular, we will develop capability to allow analysts to select document sets of interest via natural language question requests; perform sentiment estimation per document, summarizing the results as a ����������������sentiment pattern��������������� that highlights areas of positive, neutral, and negative valence; combine sentiment patterns with relevance scores from the question answering system, document body text, time, and any other available document properties to construct ����������������narrative threads��������������� that connect documents over time that appear to discuss a common topic or evolving narrative; visualize narrative threads in a web-based system, with a user interface that supports flexible control over both the construction of the threads, and the ability to explore threads, documents, and document sentiment on demand.

Our goal in this project is to focus on two related tasks. First, we will extend work from last year������������������s (2014-2015) SAS-NCSU MRA research project to study predictive analytics for narrative threads: the ability to predict how a sequence of narrative events could unfold in the future, based on past and current events; algorithms to determine how different past choices would have effect on current and future states; and visualization and user interface tools and techniques to allow users to explore the predictive space in efficient and effective ways. Second, we will collaborate with the deep learning group to study the problem of visualizing nodes and relationships in the layers of a convolution neural networks (CNN). Visualization may allow researchers to gain important insights into how different nodes in the layers of a CNN are contributing to their final output.

Situation Awareness (SA) is a process highly dependent on the analyst or operator who is attempting to defend either a physical attack (e.g., missile attack) or a cyber attack. That is, SA is ultimately a mental process of human beings. In this project we will develop an integrated end-to-end (spanning the whole ?life cycle?) cyber SA solution to fill the gap between machine information processing and analysts? mental processes. Our bridging innovations include adding the bridges or ?missing links? between the analysts? mental processes and existing machine level attack/intrusion analysis tools; between human-comprehensible situation representation and algorithmic data structures; between brainside decision making and machine-side data aggregation; and between uncertainty/risk management and largely deterministic machine state transitions. In sum, our solution adds the new algorithms and techniques that are needed for the machine SA system to work in concert with the human SA system. Our solution also integrates situation recognition, impact assessment, causality analysis, trend analysis, and assessment of system assurance The NCSU participants will focus on the development of multi-level information fusion in the cyber world, VM-based automated vulnerability diagnosis of unknown cyber vulnerabilities, and application of video game technology to bridge the gap between the cyber and the human worlds.

DO3 Task Order 2.4 Narrative Processing

A significant challenge for Cyber-Physical systems is incorporating human judgment into a complex analysis process. In a fully automated analysis, results and their justification can be difficult for users to understand and trust; a more effective approach is to support interactive construction and incremental modification of findings. We are designing and implementing a visualization assistant, ViA, that supports mixed-initiative interaction to collaborate with an analyst during visualization construction. Mixed-initiative approaches allow the computer and the user to share their expertise: for example, large-scale computation, search, and query processing, performed by the visualization system, together with the application of domain knowledge and expertise, as well as suggestions or constraints based on the data and tasks, provided by an analyst. Extensions of ViA will concentrate on improved higher-level support an analyst’s current workflow and mental models. One common criticism of past visualization efforts has been that we provided tools and asked the analysts to fit their problems to our tools, rather than building tools for the analysts’ problems.

This project has the overall objective of producing a novel intuitive decision making system to support a wide variety of military missions. The project has two components: assigning sentiment to social media, including representation of uncertainty, and connecting it to topic labels; and organizing information with respect to sensemaking in the areas of noticing and bracketing.