More dirt on the filter is proof of more air to the motor.
Dirt doesn't just 'find it's way in', it's sucked in with the surrounding air.
The Port Tube (restrictor) is designed to let air in only at certain pulse frequencies.
This is how it limits air intake.
It has nothing to do with the opening size, but with the tuned length of the port tube.
Removing the Port Tube lowers the resonant frequency of the Air box, allowing air to enter over a wider range of pulse frequencies.
Removing just the restrictor alone will do NOTHING to improve performance.
You must do all mods together as a 'system' or you will be wasting your time.
You can fight it all you want, but if you want a clean running motor with the best possible throttle response at all rpms, you have to remove the restictor.
I use a K&N air filter just because the things are (in my opinion) so easy to clean and maintain, so I service the filter after every 1-2 rides anyway, and I have seen that it is a bit dirtier now that the airbox port tube is gone, but I also found a noticeable difference in performance.... definitely worth more frequent air filter cleanings.