Intelligent Computing & Codesign Lab

Scientists want faster, more powerful high-performance supercomputers to simulate complex physical, biological, and socioeconomic systems with greater realism and predictive power. In May, Los Alamos scientists doubled the processing speed of the previously fastest computer.Roadrunner, a new hybrid supercomputer, uses a video game chip to propel performance to petaflop/s speeds capable of more than a thousand trillion calculations per second.”The computer is a speed demon. It will allow us to solve tremendous problems,” said Thomas DAgostino, head of the National Nuclear Security Administration, which oversees nuclear weapons research and maintains the warhead stockpile.The computer might also have many medical and science applications, including developing biofuels or discovering drug therapies.

Continue reading: Computing::Worlds Fastest Computer::Los Alamos Lab.

There is a paper that is very closely related to our work on RFV. Question is how do they differ?

First terminology. Microarchitecture-level masking is  hardware-dependent portion whereas architecture-level masking is software-dependent. Then AVF captures both, while their PVF is claimed to capture only the latter. “Therefore,  PVF  is  impacted  only  by changes  to  the  binary  or  to  input  data  and  not  by changes in hardware.” That’s it. It does not really say about how to statically estimate the PVF without actually running the program.

But this paper is important in that it establishes the needs for our static estimation techniques.

The architectural vulnerability factor (AVF) of a hardware structure is the probability that a fault in the structure will affect the output of a program. AVF captures both microarchitectural and architectural fault masking effects; therefore, AVF measurements cannot generate insight into the vulnerability of software independent of hardware. To evaluate the behavior of software in the presence of hardware faults, we must isolate the software-dependent (architecture-level masking) portion of AVF from the hardware-dependent (microarchitecture-level masking) portion, providing a quantitative basis to make reliability decisions about software independent of hardware. In this work, we demonstrate that the new program vulnerability factor (PVF) metric provides such a basis: PVF captures the architecture-level fault masking inherent in a program, allowing software designers to make quantitative statements about a program’s tolerance to soft errors. PVF can also explain the AVF behavior of a program when executed on hardware; PVF captures the workload-driven changes in AVF for all structures. Finally, we demonstrate two practical uses for PVF: choosing algorithms and compiler optimizations to reduce a program’s failure rate.

by Sridharan, V.;   Kaeli, D.R.;
Dept. of Electr. & Comput. Eng., Northeastern Univ., Boston, MA

This paper appears in: High Performance Computer Architecture, 2009. HPCA 2009. IEEE 15th International Symposium on

via IEEE Xplore – Eliminating microarchitectural dependency from Architectural Vulnerability.

•HTM(Hierarchical Temporal Memory) : Not programmed & not different algorithms for different problem.

1) Discover cause

– Find relationships at inputs.

– Possible cause is called “belief”.

2) Infer causes of novel input

– Inference : Similar to pattern recognition

– Ambiguous -> Flat.

– HTMs handle novel input both during inference & training

3) Make predictions

– Each node store sequences of patterns

+ current input -> Predict what would happen next.

4) Direct behavior : Interact with world.

How do HTMs discover and infer causes?

Why is a hierarchy important?

1) Shared representations lead to generalization and storage efficiency.

2) The hierarchy of HTM matches the spatial and temporal hierarchy of the real world.

3) Belief propagation ensures all nodes quickly reach the best mutually compatible beliefs.

– Belief propagation calculates the marginal distribution for

each unobserved node, conditional on any observed nodes.

4) Hierarchical representation affords mechanism for attention

How does each node discover and infer causes?

Assigning causes.

Most common sequence of pattern are assigned.

Assigned causes are used for prediction, behavior etc.

Why is time necessary to learn?

•Pooling(many-one)  method

– Overlap

Several images of watermelons are overlapped in one picture

– Learning of sequence : HTM uses this way.

PCNN (Pulse-Coded Neural Networks) : A modeled network system which is for the evaluation of a biology-oriented image processing, usually performed on general-purpose computers, e. g. PCs or workstations.
SNNs(Spiking Neural Networks) : A neural network model. In addition to neuronal and synaptic state, SNNs also incorporate the concept of time into their operating model.

SEE(Spiking Neural Network Emulation Engine) : A field-programmable gate array(FPAG) based emulation engine for spiking neurons and adaptive synaptic weights is presented, that tackles bottle-neck problem by providing a distributed memory architecture and a high bandwidth to the weight memory.

PCNN – Operated by PC & workstation -> Time consuming
– Because of bottle-neck : sequential access weight memory

FPGA SEE – Distribute memory
– High bandwidth weight memory
– separating calculation neuron states & network topology

SNNs or PCNNs 1. Reproduce spike or pulse.
2. Perform some problems such as vision tasks.

Problem of Large PCNNs 1. Calculation steps.
2. Communication resources.
3. Load balancing.
4. Storage capacity.
5. Memory bandwidth.

Spiking neuron model with adaptive synapses.

Non-leaky integrate-and-fire neuron(IFN)

,    ,  

Overview of the SEE architecture

Overview of SEE architecture

A. Simulation control(PPC2) 1. Configuration of network

2. Monitoring of network parameter.

3. Administration of event-list.

– Two event-lists  :  DEL(Dynamic Event-List) includes all excited neurons that receive

a spike or an external input current.

FEL(Fire Event-List) stores all firing neurons that are in a spike

sending state and the corresponding time values when the

neuron enters the spike receiving state again.

B. Network Topology Computation(NTC)

– Topology-vector-phase : The presynaptic activity is determined for each excited neuron.

– Topology-update-phase : The tag-fields are updated according to occurred spike start-events or spike stop-events.

C. Neuron State Computation(NSC)

– Neuron-spike-phase : It is determined if before the next spike stop-event an excited

neuron will start to fire.

– Neuron –update-phase

– Bulirsch_Stoer method of integration.(MMID, PZEXTR)

– Modified-midpoint integration(MMID)

– Polynomial extrapolation(PZEXTR)

PCB of spiking neural network emulation engine

Performance analysis

– Reference

Emulation Engine for Spiking Neurons and Adaptive Synaptic Weights by H. H. Hellmich, M. Geike, P. Griep, P. Mahr, M. Rafanelli and H. Klar.