A Quantum Family Algebra for a given Lie algebra g and a g-module V is defined as a subalgebra of g-invariant elements in the tensor product of U(g) and End V. For g=sl(2) the structure of such algebras will be described.