For data given on a nonlinear space, like angles, symmetric positive matrices, the sphere, or the hyperbolic space, there is often enough structure to form a Riemannian manifold. We present the Julia package Manifolds.jl, providing a fast and easy to use library of Riemannian manifolds and Lie groups. We introduce a common interface, available in ManifoldsBase.jl, with which new manifolds, applications, and algorithms can be implemented. We demonstrate the utility of Manifolds.jl using Bézier splines, an optimization task on manifolds, and a principal component analysis on nonlinear data. In a benchmark, Manifolds.jl outperforms existing packages in Matlab or Python by several orders of magnitude and is about twice as fast as a comparable package implemented in C++.