Yes, provided you know which house you own. The parish carries 53 listed buildings, a single conservation area covering Pevensey and Westham, and 3 scheduled monuments in the surrounding area. Most of the housing stock is post-war and not directly caught by any of that. A smaller share of the stock is heavily caught by all of it. Which side of that split your address sits on decides everything about what you can and can't do to it.
Listing is the heavier constraint. There are 53 listed buildings in and around the village: 2 at Grade I, 4 at Grade II, and 47 at Grade II. Grade I is the top tier, reserved for buildings of exceptional interest, and comes with the tightest control. Grade II covers the next tier down, and Grade II is the standard listing that catches the majority of the total. If a house is listed at any grade, virtually any change that affects character requires listed building consent alongside any ordinary planning permission. That covers windows, doors, internal walls, chimneys, fireplaces, staircases, and often the external paint colour. It also covers things people don't think of as alterations: replacing a boiler flue, drilling through a wall for a cable, changing the front door furniture. Listed building consent is a formal process run through the Wealden district planning department, with input from Historic England on the higher-grade properties. Refusal is possible. Enforcement on unconsented work is possible. The listing is not decorative, and it comes with the house.
The conservation area covers Pevensey and Westham as a single designated zone, which broadly maps to the historic village and its adjoining Westham parish. Living inside a conservation area does not require listed building consent (unless the specific house is also listed) but it does mean tighter planning scrutiny for anything visible from the public realm: extensions, dormers, cladding, boundary treatments, tree work above a set trunk size, and any change that affects the group character. Some permitted development rights that would apply to an unrestricted post-war house are trimmed in a conservation area, which typically means an ordinary Google-search extension proposal turns into a formal planning application. The three scheduled monuments in the surrounding area include the castle and other archaeologically significant sites, and any development that would affect their setting is a separate consideration in its own right.
For anyone buying a standard modern house outside the conservation area and off the listed register, the practical picture is close to normal permitted development. A rear extension within the standard dimensions, a loft conversion within the standard volume, a garden building within the standard footprint, and the usual internal changes are all in scope without a full planning application, though building control still applies. For anyone buying inside the conservation area or a listed property, the practical picture is different: assume every change needs a consultation with the district planning team and probably a formal application, and budget both the time and the consultant cost accordingly. The pre-application service at Wealden is worth using before spending money on drawings. Discovering the constraint at the enforcement stage is the expensive way to learn it.
The overall verdict on freedom to alter is average, and average is fair. Most of the parish is unconstrained. A small, geographically clustered slice of it is heavily constrained. What you can do to the house is decided by which slice it sits in, not by the parish average.