entering your property and digging in your yard are two distinct issues. You live in Englandand the lawsmay vary. But in the US (barring specific legislation to thecontrary) a person may enter in effect enter any outdoor space or indoor space open to the public. This is unless notice to the contrary is given. That is why thempesky salesmen can come to your door.

Let us presume your garden does not look like the Nevada desert post destruction of a building. There is a term of art called 'a reasonably prudent person'. This comes up when a court is determining whether the action taken is legitimate. A RPP would not look at a nice garden and think tear it up. But a RPP upon hearing of a find of 'abandoned property', might feel free to lay claim to that property.

There is a Supreme Court Case that declares traSH to be abandoned property. It is what allows LE to search your trash without a warrant. It is also a good reason tonot put any personal info in your trash. But if you throw your trash bags out to the stoop, then place your Van Gogh next to them (while you bring the truck around) a RPP could interpret that as abandoned property identicle to the bags next to it. (The last example is taken from a real case in Illinois).