Replacing triangulated categories with something better
My opinion, and that of many other people although not of everyone, is that the "correct" notion is that of stable ∞-category.
Now, this is not a category in the strictest sense, rather a generalization of the notion of category known as an (∞,1)-category, or ∞-category for short, where to any pair of objects $x,y$ there is an associated homotopy type $\mathrm{Map}_{\mathcal{C}}(x,y)$, usually called the mapping space. You can get a category from that datum by taking the connected components $\pi_0\mathrm{Map}_{\mathcal{C}}(x,y)=:[x,y]$. The resulting category is called the homotopy category $h\mathcal{C}$, and can be seen as the best approximation you can give of an ∞-category using an ordinary category.
You can talk about limits and colimits in an ∞-category, and in fact pretty much all of classical category theory goes through in this more general setting without problems (although with the occasional very important modification). Then you can say that an ∞-category $\mathcal{C}$ is stable if it satisfies the two following conditions:
It has a zero object (i.e. an object $0$ such that $\mathrm{Map}(x,0)$ and $\mathrm{Map}(0,x)$ are contractible for every $x\in\mathcal{C}$).
It has all pullbacks and pushouts and a square (i.e. a diagram of the form $[1]\times [1]\to\mathcal{C}$) is cartesian iff it is cocartesian.
As you can see, it is a fairly simple definition. It can be rephrased in a few equivalent ways, some of which are rather easy to check. This notion has a few very important properties:
For every stable ∞-category $\mathcal{C}$, the homotopy category $h\mathcal{C}$ has a canonical triangulated structure.
All triangulated categories that actually show up in mathematical practice usually come equipped with a specific stable enrichment (i.e. a stable ∞-category whose homotopy category is the triangulated category you were thinking about). In a few cases, the stable ∞-category is actually easier to define.
There are examples of triangulated categories that do not come from a stable ∞-category. All the examples tend to look unnatural, and we would very much like a definition that excludes them.
In stable ∞-categories, a lot of the theorems that one would expect to be naively true for triangulated categories are actually true. For example, cones are functorial, and you can define the algebraic K-theory of a stable ∞-category (while you cannot do so for a triangulated category!), obtaining the expected results (e.g. the algebraic K-theory of the stable ∞-category of perfect complexes over a ring is exactly the algebraic K-theory of the ring).
More abstractly, stable ∞-categories work well in families. For example, the functor sending a scheme $X$ to the stable ∞-category of perfect complexes over $X$ is a fppf sheaf (for an appropriate notion of sheaf of ∞-categories). This is not true for the corresponding triangulated categories!
Just a quick observation that everyone should know but that I seem to have been the first to notice is that the fill-in axiom, the one where the non-functoriality of cones is most glaringly visible, is actually redundant. It is implied by the so called octahedral axiom, which is purely an axiom about the behavior of exact triangles with respect to composition, together with the less substantial axioms. That is, with the usual nomenclature, Verdier's (TR1), (TR2), and (TR4) impy his (TR3). See Section 2 of http://www.math.uchicago.edu/~may/PAPERS/AddJan01.pdf. It is remarkable how very much mathematics can be done starting with just these achingly simple axioms, which are designed to be applicable on the homotopy category level in the most general possible setting. Of course, more restrictive and elaborate contexts before passage to homotopy categories are often needed as well (starting from model categories or from $\infty$ categories or from $\cdots$ according to taste and need --- personally, I believe in being eclectic, not in ``correct'' notions).
I would propose a different alternative. It is the theory of (Grothendieck's) derivators, at least the stable variant. It was also developed by Heller under the name "homotopy theories" and very much related to Keller's "towers of triangulated categories" and to Franke's "systems of triangulated diagram categories". Roughly, to a base triangulated category one adds all homotopy limits and colimits, essentially adjoints (that arise as Kan extensions) to the constant diagram with values in a triangulated category of "coherent diagrams".
Once you have a derivator, to be stable is a property, not a structure on it. This property is reasonably easy to check in the main examples and distinguishes stable phenomena. Stability immediately yields a collection of distinguished triangles satisfying the usual axioms. Also octahedra and higher triangles are produced by this property and they behave in a right way form the homotopical point of view, implicitly satisfying the universal properties up to homotopy that defines them.
A very nice exposition is the paper by Groth "Derivators, pointed derivators, and stable derivators" (Algebraic & Geometric Topology 13 (2013), 313-374)
http://www.math.uni-bonn.de/~mrahn/publications/groth_derivators.pdf
For some people this notion is simpler than $\infty$-categories and encompasses the work recently done with just the axioms mentioned by Peter May in his answer together with the existence of arbitrary coproducts.
The idea of Grothendieck was to express the deep meaning behind the notion of homotopy. To the extent he achieved this is debatable. But, the flexibility of stable derivators for extending homotopical constructions in triangulated categories without making recourse to model categories is one of the features some people may find useful.