Not actively picking people up when you have the ability to hang signs all around the state and just waiting for people to come to you stinks of trap.
Any group that had the ability to spread signs over such a large area as they show the routes on the sign maps (the entire state), had to have been active for quite awhile, that would be months of work to put up the signs all over. Pretty much every town has a railroad line going through it. So I would suspect they went through Woodbury at some point. Now the area that was "safe" was only a few blocks, but still seems like the scouts hanging signs would have noticed something. If they really are offering sanctuary to anyone who shows up at their place while not actively searching for survivors, one would think if they stumbled upon a large group of survivors they would at least make an attempt to contact them. I mean they want people to come to them, and are out advertising it, and they see some people, why hide from them?
Of course like I said the first time, I think the issue is more that all this is just too much logic to expect from a TV show. I think Terminus was not foreshadowed in early episodes because it had not existed, as it doesn't exist in the GN. While there is a lot of speculation of Terminus turning out to be a few different things from the GN, there was no signs leading survivors from the prison to some sanctuary. So I suspect while Woodbury and the prison were happening, they hadn't actually written in Terminus for the TV show. So little things like the Terminus scouts not contacting the other settlements would not have come up as there wasn't a Terminus at the time.
And you know for a fact that these Terminus people have put up signs "all around the state", or are you just assuming that from some black lines traced on a map?
You can assume all you want, but like I said, we have no idea how long those signs have been there. How many people are at Terminus, 10, 50, 500? When did they start putting up the signs, last year or last month? Did they have enough people to start putting out signs in every direction, or did they go down one spur at a time? Was the spur the various groups are following now one of the first ones they posted signs on or one of the last ones?
We'll find out what the deal is eventually, and you may very well be correct, but until then your "logic" is full of assumptions.
ETA: What would really suck, and would be right up the Troll King's (Kirkman) alley, would be for them to focus all their attention on Terminus, walk all that way, and end-up finding a place overrun with walkers.