My guess is that the problem i
 

My guess is that the problem is not directly removing the patch; the problem is that you have external faces and you want to make them internal?

If so you could try using the stitchMesh utility which requires the two sides of the baffle to be in different patches. Or just remesh.