# Implicit parallelism

> Mediated Wiki article. Canonical URL: https://mediated.wiki/source/Implicit_parallelism
> Markdown URL: https://mediated.wiki/source/Implicit_parallelism.md
> Source: https://en.wikipedia.org/wiki/Implicit_parallelism
> Source revision: 1351342089
> License: Creative Commons Attribution-ShareAlike 4.0 International (https://creativecommons.org/licenses/by-sa/4.0/)

Inherent parallelism in expressed computation

This article needs more citations. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed. Find sources: "Implicit parallelism" – news · newspapers · books · scholar · JSTOR (February 2024) (Learn how and when to remove this message)

In [computer science](/source/Computer_science), **implicit parallelism** is a characteristic of a [programming language](/source/Programming_language) that allows a [compiler](/source/Compiler) or [interpreter](/source/Interpreter_(computing)) to automatically exploit the [parallelism](/source/Parallel_computing) inherent to the computations expressed by some of the language's constructs. A pure implicitly parallel language does not need special directives, operators or functions to enable parallel execution, in contrast to [explicit parallelism](/source/Explicit_parallelism).

Programming languages with implicit parallelism include [Axum](/source/Axum_(programming_language)), [Binary Modular Dataflow Machine](/source/Binary_Modular_Dataflow_Machine) (BMDFM), [High Performance Fortran](/source/High_Performance_Fortran) (HPF), [Id](/source/Id_(programming_language)), [LabVIEW](/source/LabVIEW), [MATLAB](/source/MATLAB) M-code, [NESL](/source/NESL), [Single Assignment C](/source/SAC_programming_language) (SAC), Streams and Iteration in a Single Assignment Language ([SISAL](/source/SISAL)), [Z-level programming language](/source/Z-level_programming_language) (ZPL), and pH.[1]

## Example

If a given problem involves performing the same operation on a group of numbers (such as taking the [sine](/source/Sine) or [logarithm](/source/Logarithm) of each in turn), a language that provides implicit parallelism might allow writing the instruction thus:

numbers = [0 1 2 3 4 5 6 7];
result = sin(numbers);

The compiler or interpreter can calculate the sine of each element independently, spreading the effort across multiple processors if available.

## Advantages

A programmer that writes implicitly parallel code does not need to worry about task division or process communication, focusing instead on the problem that their program is intended to solve. Implicit parallelism generally facilitates the design of parallel programs and therefore results in a substantial improvement of programmer productivity.

Many of the constructs necessary to support this also add simplicity or clarity even in the absence of actual parallelism. The example above, of [list comprehension](/source/List_comprehension) in the sin() function, is a useful feature in of itself. By using implicit parallelism, languages effectively have to provide such useful constructs to users simply to support required functionality (a language without a decent [for loop](/source/For_loop), for example, is one few programmers will use).

## Disadvantages

Languages with implicit parallelism reduce the control that a programmer has over the parallel execution of a program, resulting sometimes in suboptimal [parallel efficiency](/source/Parallel_efficiency). The makers of [Oz](/source/Oz_(programming_language)) language also note that their early experiments with implicit parallelism showed that implicit parallelism made [debugging](/source/Debugging) difficult and [object models](/source/Object_model) unnecessarily awkward.[2]

A larger issue is that every program has some parallel and some serial logic. Binary [input/output](/source/Input%2Foutput) (I/O), for example, requires support for such serial operations as Write() and Seek(). If implicit parallelism is desired, this creates a new requirement for constructs and keywords to support code that cannot be [threaded](/source/Thread_(computing)) or distributed.

## Notes

1. **[^](#cite_ref-1)** Nikhil, Rishiyur; [Arvind](/source/Arvind_(computer_scientist)) (February 20, 2024). *Implicit Parallel Programming in pH*. Morgan Kaufmann Publishers. [ISBN](/source/ISBN_(identifier)) [978-1-55860-644-9](https://en.wikipedia.org/wiki/Special:BookSources/978-1-55860-644-9).

1. **[^](#cite_ref-2)** Haridi, Seif (June 14, 2006). "Introduction". [Tutorial of Oz](https://web.archive.org/web/20110514010946/http://www.mozart-oz.org/documentation/tutorial/node1.html#chapter.introduction) (Report). Archived from the original on May 14, 2011. Retrieved September 20, 2007.

v t e Parallel computing General Distributed computing Parallel computing Parallel algorithm Massively parallel Cloud computing High-performance computing Multiprocessing Manycore processor GPGPU Computer network Systolic array Levels Bit Instruction Thread Task Data Memory Loop Pipeline Multithreading Temporal Simultaneous (SMT) Simultaneous and heterogenous Speculative (SpMT) Preemptive Cooperative Clustered multi-thread (CMT) Hardware scout Theory PRAM model PEM model Analysis of parallel algorithms Amdahl's law Gustafson's law Cost efficiency Karp–Flatt metric Slowdown Speedup Elements Process Thread Fiber Instruction window Array Coordination Multiprocessing Memory coherence Cache coherence Cache invalidation Barrier Synchronization Application checkpointing Programming Stream processing Dataflow programming Models Implicit parallelism Explicit parallelism Concurrency Non-blocking algorithm Hardware Flynn's taxonomy SISD SIMD Array processing (SIMT) Pipelined processing Associative processing MISD MIMD Dataflow architecture Pipelined processor Superscalar processor Vector processor Multiprocessor symmetric asymmetric Memory shared distributed distributed shared UMA NUMA COMA Massively parallel computer Computer cluster Beowulf cluster Grid computer Hardware acceleration APIs Ateji PX Boost Chapel HPX Charm++ Cilk Coarray Fortran CUDA Dryad C++ AMP Global Arrays GPUOpen MPI OpenMP OpenCL OpenHMPP OpenACC Parallel Extensions PVM pthreads RaftLib ROCm UPC TBB ZPL Problems Automatic parallelization Cache stampede Deadlock Deterministic algorithm Embarrassingly parallel Parallel slowdown Race condition Software lockout Scalability Starvation Category: Parallel computing

This computer science article is a stub. You can help Wikipedia by adding missing information.

- [v](https://en.wikipedia.org/wiki/Template:Comp-sci-stub)
- [t](/source/Template_talk%3AComp-sci-stub)
- [e](https://en.wikipedia.org/wiki/Special:EditPage/Template:Comp-sci-stub)

---
Adapted from the Wikipedia article [Implicit parallelism](https://en.wikipedia.org/wiki/Implicit_parallelism) by Wikipedia contributors ([contributor history](https://en.wikipedia.org/wiki/Implicit_parallelism?action=history)). Available under [Creative Commons Attribution-ShareAlike 4.0 International](https://creativecommons.org/licenses/by-sa/4.0/). Changes may have been made.