A polytope or polytope compound is self-intersecting if it has at least two distinct elements with interiors that intersect. For the sake of this definition, a vertex's location is considered to be its interior. (For example, if a unit cube has a square pyramid excavated from it so the apex of the pyramid touches a face of the cube, the resulting polyhedron is self-intersecting.)
Non-self-intersecting polytopes are sometimes called acoptic, and polygons that fit this definition are often called simple.
Self-intersection is dependent on the definition of "interior". Skew elements generally have no defined interior, so self-intersection is not generally defined for polytopes with skew elements[note 1]. For non-skew polytopes, "interior" is technically dependent on filling method, but if two distinct filling methods agree on the definition of interior for non-self-intersecting polytopes, induction shows that they will also agree on which polytopes are self-intersecting. As there is broad agreement on filling methods for non-self-intersecting polytopes, in practice self-intersection is virtually unambiguous for non-skew polytopes in Euclidean space.
Self-intersection often creates confusion for novice polytopists as it defies visual intuition. Historically, the Kepler-Poinsot solids were only studied and characterized millennia after the Platonic solids. However, it's usually more mathematically "natural" to restrict discussion to either convex polytopes or general polytopes permitting self-intersection, rather than the intermediate category of general polytopes excluding self-intersection. That said, some polytopists have specifically sought out non-convex polytopes without self-intersection, such as in Stewart toroids and some discussions of acrohedra.