parallel computing(并行计算)：is the use of a parallel computer to reduce the time needed to slove a single computation problem.(使用并行计算机来减少解决单个问题所需的时间)
parallel computer(并行计算机)：is a multiple-processor computer system supporting parallel programming.(支持并行计算的多处理器计算机系统)
multicomputer--多计算机 centralized multiprocessors--集中式多处理器
a centrlized multiprocessors(also called a symmetrical multiprocessor or SMP--对称多处理器系统) is a more higly integrated system in which all CPUs share access to single golbal memory.This shared memory supports communication(通信) and synchronization(同步)
parallel programming (并行程序设计) ： is programming in a language that allows you to explicity indicate how different portions of computation may be executed concurrently by different processors.(使用程序设计语言显式的说明计算中不同的部分如何在不同处理器上同时执行)
MPI--Message Passing Interface(消息传递接口) ：is a standard specificatuon for message-passing libraries.(消息传递库的标准)
The MPI library supports parallel programming through message passing. It allows processors that do not share memory to coopertate in performing a parallel computation.
Data Depedence Graphs--数据相关图
Data Parallelism--数据并行性：independent tasks applying the same operation to different elements of a data set.(不相关的任务对数据集的不同元素进行相同的操作)
Functional Parallelism--功能并行性 ：independent tasks applying different operations to different data elements.(不相关的任务对数据集的不同元素进行不同的操作)
Pipelining--流水线: the ouput of one stage is the input of the next.
（数据相关图中的并行性，节点代表 任务，节点内的字母代表执行的操作，边代表任务间的相关性.(a)具有数据并行性的图，不同的工人可以同时进行操作B；(b)具有功能并行性的图，执行B,C,D操作的任务可以同时进行；(C)完全串行的相关图，但是如果每个任务都需要相同的时间，并且需要处理多个问题实例的时候，可以在处理第i个问题的操作C的同时,处理第i+1个问题的B操作以及第i+2个问题的操作A，称为流水线结构）
Data Clustering--数据聚类
data mining -- 数据挖掘
scientific data analysis -- 科学数据分析
Programming parallel computers -- 为并行计算机编程
Extend a Compiler -- 扩展编译器: One approach to the problem of programming parallel computers is to develop parallelizing compilers that can detect and exploit the parallelism in existing programs written in a sequential language.(解决并行编程的方法之一是开发并行化编译器，使其能够发现和表达现有串行语言程序中的并行性)
Extend a Sequential Programming Language -- 扩展串行编程语言：
Add a Parallel Programming Layer -- 增加并行编程层
CODE(Computationlly Oriented Display Evironment--面向计算的显示环境)
Hence(Heterogeneous Network Computing Evironment--异构网络计算环境)
These system allow the use t depict a parallel program as a directed graph,where nodes represent sequential procedures and arcs(边) represent data depedences among procedures.
Create a Parallel Lanugae -- 创造一个并行语言
HPF(high performance fortan--高性能Fortan)
HPC(high performance compututing -- 高性能计算)
The process of the operations to deliver a C program with MPI to a remote Linux server and launch to the program with dedicated configuration of MPI.
Principles:
Install PuTTY on a Windows machine and then login a remote Linux server (hpc.fafu.edu.cn ) with password-based authentication.
pscp: 将pscp放在C:\WINDOWS\SYSTEM32下
WIN+R 打开CMD
发送文件到Linux上：（如hello.c在D盘中）
输入：
pscp D:\hello.c 3156010011@hpc.fafu.edu.cn:/export/home/student/3156010011
查看man mpicc 以及man mpirun的帮助信息
The command mpirun -help gives all the options available, using these options as appropriate to better run the application and improve system health.
The basic format of mpirun is:
Mpirun [mpirun-options ...] <progname> [options ...]
Where [mpirun-options ...], the main options are as follows:
-np <np> The number of processes to load.
-p4pg <pgfile> Loads the user process as required by the pgfile file. The pgfile file describes what user processes are loaded on those nodes. The format of the file is:
The first line: <node name> <0> <user to load the process - allows the use of absolute path>
The second line: <node name> <1> <user to load the process - allows the use of absolute path>
Line n: <node name> <1> <user to load the process - allow the use of absolute path>
Where n is the number of users to load the process. The node name can be the same or different. And the user has this option, the -np option is invalid.
mpicc [option] source:
Commanly used options list:
·-c: Only compile, do not link, that is, only the target file (.o file)
·-o filename: Specifies the file name of the output, usually by default(a.out)
·-Ipath: Specify (add) header files (eg * .h) Search path (directory)
·-Lpath: Specify (increase) the search path (directory) of the library file
·-lname: With the library file libname. link
·-g: Make The object code contains the source file name, line number and other information(for program debugging)
2.The differences between Process and Thread.
The Process is running program, it provides two abstractions: one is a Logical control flow. The second is Private address space. The first abstract implantation is to slice the CPU time, each time slice is used to execute a process. When the time slice is consumed, the operating system switches the current process to another. This called a context switch.
A process which has a good number of threads, share code, data, as well as Kernel context before the thread. Each thread has its own stack and context. A process which is the same level of threads that share data and code. There is a hierarchical relationship between processes. Threads are relatively lightweight relative to the process.
Threads can communicate and coordinate through shared variables. Shared variables are less costly than message passing and network. Interprocess communication is mainly through messaging and networking.
So the key difference between the two term is that the threads are a part of a process, i.e. a process may contain one or more threads, but a thread cannot contain a process.
