# Task parallelism > Mediated Wiki article. Canonical URL: https://mediated.wiki/source/Task_parallelism > Markdown URL: https://mediated.wiki/source/Task_parallelism.md > Source: https://en.wikipedia.org/wiki/Task_parallelism > Source revision: 1237880513 > License: Creative Commons Attribution-ShareAlike 4.0 International (https://creativecommons.org/licenses/by-sa/4.0/) This article includes a list of general references, but it lacks sufficient corresponding inline citations. Please help to improve this article by introducing more precise citations. (May 2011) (Learn how and when to remove this message) Form of parallelization of computer code **Task parallelism** (also known as **function parallelism** and **control parallelism**) is a form of [parallelization](/source/Parallelization) of [computer code](/source/Source_code) across multiple [processors](/source/Central_processing_unit) in [parallel computing](/source/Parallel_computing) environments. Task parallelism focuses on distributing [tasks](/source/Task_(computing))—concurrently performed by [processes](/source/Process_(computing)) or [threads](/source/Thread_(computing))—across different processors. In contrast to [data parallelism](/source/Data_parallelism) which involves running the same task on different components of data, task parallelism is distinguished by running many different tasks at the same time on the same data.[1] A common type of task parallelism is [pipelining](/source/Pipeline_(computing)), which consists of moving a single set of data through a series of separate tasks where each task can execute independently of the others. ## Description In a multiprocessor system, task parallelism is achieved when each processor executes a different thread (or process) on the same or different data. The threads may execute the same or different code. In the general case, different execution threads communicate with one another as they work, but this is not a requirement. Communication usually takes place by passing data from one thread to the next as part of a [workflow](/source/Workflow).[2] As a simple example, if a system is running code on a 2-processor system ([CPUs](/source/CPU) "a" & "b") in a [parallel](https://en.wiktionary.org/wiki/parallel) environment and we wish to do tasks "A" and "B", it is possible to tell CPU "a" to do task "A" and CPU "b" to do task "B" simultaneously, thereby reducing the [run time](/source/Run_time_(program_lifecycle_phase)) of the execution. The tasks can be assigned using [conditional statements](/source/Conditional_(programming)) as described below. Task parallelism emphasizes the distributed (parallelized) nature of the processing (i.e. threads), as opposed to the data ([data parallelism](/source/Data_parallelism)). Most real programs fall somewhere on a continuum between task parallelism and data parallelism.[3] **Thread-level parallelism** (**TLP**) is the [parallelism](/source/Parallel_computing) inherent in an application that runs multiple [threads](/source/Thread_(computer_science)) at once. This type of parallelism is found largely in applications written for commercial [servers](/source/Server_(computing)) such as databases. By running many threads at once, these applications are able to tolerate the high amounts of I/O and memory system latency their workloads can incur - while one thread is delayed waiting for a memory or disk access, other threads can do useful work. The exploitation of thread-level parallelism has also begun to make inroads into the desktop market with the advent of [multi-core](/source/Multi-core) microprocessors. This has occurred because, for various reasons, it has become increasingly impractical to increase either the clock speed or instructions per clock of a single core. If this trend continues, new applications will have to be designed to utilize multiple threads in order to benefit from the increase in potential computing power. This contrasts with previous microprocessor innovations in which existing code was automatically sped up by running it on a newer/faster computer. ## Example The [pseudocode](/source/Pseudocode) below illustrates task parallelism: program: ... if CPU = "a" then do task "A" else if CPU="b" then do task "B" end if ... end program The goal of the program is to do some net total task ("A+B"). If we write the code as above and launch it on a 2-processor system, then the runtime environment will execute it as follows. - In an [SPMD](/source/SPMD) (single program, multiple data) system, both [CPUs](/source/CPU) will execute the code. - In a parallel environment, both will have access to the same data. - The "if" clause differentiates between the CPUs. CPU "a" will read true on the "if" and CPU "b" will read true on the "else if", thus having their own task. - Now, both CPU's execute separate code blocks simultaneously, performing different tasks simultaneously. Code executed by CPU "a": program: ... do task "A" ... end program Code executed by CPU "b": program: ... do task "B" ... end program This concept can now be generalized to any number of processors. ## Language support Task parallelism can be supported in general-purpose languages by either built-in facilities or libraries. Notable examples include: - Ada: Tasks (built-in) - C++ (Intel): [Threading Building Blocks](/source/Threading_Building_Blocks) - C++ (Intel): [Cilk Plus](/source/Cilk_Plus) - C++ (Open Source/Apache 2.0): [RaftLib](/source/RaftLib) - C, C++, Objective-C, Swift (Apple): [Grand Central Dispatch](/source/Grand_Central_Dispatch) - D: [tasks](/source/Task_(computing)) and [fibers](/source/Fiber_(computer_science)) - Delphi (System.Threading.TParallel) - Go: [goroutines](/source/Goroutine) - Java: [Java concurrency](/source/Java_concurrency) - .NET: [Task Parallel Library](/source/Task_Parallel_Library) Examples of fine-grained task-parallel languages can be found in the realm of [Hardware Description Languages](/source/Hardware_Description_Language) like [Verilog](/source/Verilog) and [VHDL](/source/VHDL). ## See also - [Algorithmic skeleton](/source/Algorithmic_skeleton) - [Data parallelism](/source/Data_parallelism) - [Fork–join model](/source/Fork%E2%80%93join_model) - [Parallel programming model](/source/Parallel_programming_model) ## References 1. **[^](#cite_ref-1)** Reinders, James (10 September 2007). ["Understanding task and data parallelism"](https://www.zdnet.com/article/understanding-task-and-data-parallelism/). *ZDNet*. Retrieved 8 May 2017. 1. **[^](#cite_ref-2)** Quinn, Michael J. (2007). *Parallel programming in C with MPI and openMP* (Tata McGraw-Hill ed.). New Delhi: Tata McGraw-Hill Pub. [ISBN](/source/ISBN_(identifier)) [978-0070582019](https://en.wikipedia.org/wiki/Special:BookSources/978-0070582019). 1. **[^](#cite_ref-3)** Hicks, Michael. ["Concurrency Basics"](http://www.cs.umd.edu/class/fall2013/cmsc433/lectures/concurrency-basics.pdf) (PDF). *University of Maryland: Department of Computer Science*. Retrieved 8 May 2017. v t e Parallel computing General Distributed computing Parallel computing Parallel algorithm Massively parallel Cloud computing High-performance computing Multiprocessing Manycore processor GPGPU Computer network Systolic array Levels Bit Instruction Thread Task Data Memory Loop Pipeline Multithreading Temporal Simultaneous (SMT) Simultaneous and heterogenous Speculative (SpMT) Preemptive Cooperative Clustered multi-thread (CMT) Hardware scout Theory PRAM model PEM model Analysis of parallel algorithms Amdahl's law Gustafson's law Cost efficiency Karp–Flatt metric Slowdown Speedup Elements Process Thread Fiber Instruction window Array Coordination Multiprocessing Memory coherence Cache coherence Cache invalidation Barrier Synchronization Application checkpointing Programming Stream processing Dataflow programming Models Implicit parallelism Explicit parallelism Concurrency Non-blocking algorithm Hardware Flynn's taxonomy SISD SIMD Array processing (SIMT) Pipelined processing Associative processing MISD MIMD Dataflow architecture Pipelined processor Superscalar processor Vector processor Multiprocessor symmetric asymmetric Memory shared distributed distributed shared UMA NUMA COMA Massively parallel computer Computer cluster Beowulf cluster Grid computer Hardware acceleration APIs Ateji PX Boost Chapel HPX Charm++ Cilk Coarray Fortran CUDA Dryad C++ AMP Global Arrays GPUOpen MPI OpenMP OpenCL OpenHMPP OpenACC Parallel Extensions PVM pthreads RaftLib ROCm UPC TBB ZPL Problems Automatic parallelization Cache stampede Deadlock Deterministic algorithm Embarrassingly parallel Parallel slowdown Race condition Software lockout Scalability Starvation Category: Parallel computing --- Adapted from the Wikipedia article [Task parallelism](https://en.wikipedia.org/wiki/Task_parallelism) by Wikipedia contributors ([contributor history](https://en.wikipedia.org/wiki/Task_parallelism?action=history)). Available under [Creative Commons Attribution-ShareAlike 4.0 International](https://creativecommons.org/licenses/by-sa/4.0/). Changes may have been made.