PasswordMaker Forums

Firefox/SeaMonkey/Mozilla/Netscape/Flock Browser Extension => Feature Requests / Enhancements => Topic started by: adamspiers on December 01, 2006, 05:25:01 PM

Title: subdomain URL matching doesn't work and other issues
Post by: adamspiers on December 01, 2006, 05:25:01 PM
Hi Eric,

Thanks a lot for a great extension.  I've read a LOT of the help available on the website, but I'm still having problems understanding how the URL matching algorithm works, and I find other aspects of the interface quite confusing.  This long post is not a rant or complaint, I just want to help you make a great extension even better :-)

For example, I want to apply a modifier to multiple subdomains of the same domain:

   subdomain1.mydomain.com
   subdomain2.mydomain.com

etc. regardless of the protocol/port/path/anchor/query.  I have been testing with 1.6.1.  I thought it would be enough to create an account with the name 'mydomain.com' which had the modifier set, but this never matched - all subdomains used the default settings.  I also set 'Use the following URL to calculate the generated password' to 'mydomain.com' but this didn't help either.  It's not clear to me which one is supposed to be used in the matching process - 'Name' sounds like it's just a label, and 'Use the following URL to calculate the generated password' sounds like it only affects calculation (not matching) of the current browser URL with this account.

The only way I ever could get it to match and apply the modifier was by explicitly adding a regexp pattern:

   https?://[^/]+\.mydomain\.com/.*

to the pattern list entitled 'Activate auto-population when the URL of a webpage matches any of the following patterns', even though I don't want auto-population to occur!  I only want population when I press the CoolKey (i.e. non-automatic - I suggest you change the sentence above it to read "Population can also activated using CoolKey", since it's confusing to say that *automatic* population can be activated *manually*).

Anyway, I hope you will say that this is a bug in the matching logic, because it would be deeply inconvenient to have to create a wildcard or regexp for every single account I create.

Nor was it clear to me whether the pattern list controlled whether population was automatic for only the extra fields in "Advanced Auto-Populate", or also for the username/password fields.  What is supposed to happen if 'Automatically populate username and password fields for sites that match this URL' is unchecked, but one of the wildcards/regexps matches this URL?  The wording suggests that in this case username/password fields will be automatically populated.

Furthermore, there really need to be at least two pattern lists per account, not just one.  Why?  Because there needs to be one to determine whether the account matches the current browser URL or not (and hence whether the account's settings should be applied), and then one to decide whether population of the custom fields in 'Advanced Auto-Populate' is automatic or not, so that for example, from a single account you could have manual population (CoolKey only) of

      http://subdomain1.mydomain.com/login (http://subdomain1.mydomain.com/login)
      http://subdomain2.mydomain.com/login (http://subdomain2.mydomain.com/login)
      http://myotherdomain.com/login (http://myotherdomain.com/login)

but automatic population of

      http://subdomain3.mydomain.com/specific/pa...rofile-settings (http://subdomain3.mydomain.com/specific/path/to/my-profile-settings)

using custom fields (for phone number, address etc.)

If two pattern lists is asking too much, please bear in mind that URL matching the account is more important than controlling automatic/manual population, so the pattern list should affect matching.  You could easily move the automatic/manual population to be an option available in each custom field.

(In a perfect world you could even have multiple sets of custom fields per account, but that's probably going too far.)

Hope this is useful.  If you fixed the matching bug, improved the wording in the dialog boxes, and made a better distinction between options which control matching vs. options which control automatic/manual, you'd be well on the way to a perfect extension :-)

Thanks,
Adam
Title: subdomain URL matching doesn't work and other issues
Post by: adamspiers on February 19, 2007, 12:25:33 AM
Any ideas on this?  Of the 8 bugs and feature requests I posted in December, this is the most urgent, as it makes multiple accounts damn awkward to use, especially for people who are concerned about security and don't know how to write safe regexps.
Title: subdomain URL matching doesn't work and other issues
Post by: Miquel 'Fire' Burns on February 19, 2007, 03:03:52 AM
Here's some extra knowledge for your RegEx
Putting something like this will do a branching of different subdomains
Code: [Select]
(subdomain1\.|subdomain2\.|www\.)?
The complete RegEx may look like this:
Code: [Select]
https?://(subdomain1\.|subdomain2\.|www\.)?mydomain\.com/.*
The subdomain part can be reduced by combining the 1 and 2 (the code is like this:
Code: [Select]
subdomain[12]Though that last bit may not apply to you because the example just happens to be numbered, but the real life situation may not work this way.
Title: subdomain URL matching doesn't work and other issues
Post by: Eric H. Jung on February 21, 2007, 01:52:40 AM
Adam,
Can you please distill your suggestions into some concise bullet points? I read your initial post about 5 times and am having a hard time focusing. I get the idea of two pattern sets, although I don't understand why you'd want to apply advanced auto-populate to patterns other than the ones specified on the URLs page.

thanks,
eric
Title: subdomain URL matching doesn't work and other issues
Post by: adamspiers on February 21, 2007, 09:37:20 PM
Quote from: miquelfire
Here's some extra knowledge for your RegEx
[...]

miquelfire - thanks a lot for trying to help, however creating the right regexp is not the issue here (I have been programming complex regular expressions for over 10 years now!) - in fact the core of the issue is that I believe I should not have to use any regexps or patterns to achieve what I want.

Quote from: Eric H. Jung
Adam,
Can you please distill your suggestions into some concise bullet points? I read your initial post about 5 times and am having a hard time focusing. I get the idea of two pattern sets, although I don't understand why you'd want to apply advanced auto-populate to patterns other than the ones specified on the URLs page.

thanks,
eric

Eric,

I did already put a lot of effort into trying to communicate this issue across in as concise and clear a manner as possible, but I obviously failed since neither of you understand what I meant!  It's a difficult one to explain, let me try again...

Imagine I want to register for a website http://www.acme.com/ (http://www.acme.com/) but the registration process at https://login.acme.com/ (https://login.acme.com/) only allows letters and digits in the password.  So I open the PasswordMaker window, select the Accounts tab, and create a new account.  In the URLs tab I see it has automatically detected the correct value of 'acme.com' for "Use the following URL to calculate the generated password".  So far so good.  I click on the "Extended" tab and change the "Characters" field so it only contains letters and digits.  I click "OK", see that my new account has appeared in the list, and click "Close".  All is well, right?  Wrong!

PROBLEM 1: Creating an account has absolutely no effect without explicitly adding a pattern or regexp to match the site against.

I thought (perhaps mistakenly) that this was the point of the "URLs" tab in the "Defaults" account - I have only the "Domain" checkbox ticked, so I thought that if I created an account for acme.com, its settings would take effect on any subdomain, such as login.acme.com.  No?

There are several other problems, but this is the biggest, so for sake of simplicity shall we try to deal with this one first?

Thanks for your time,
Adam
Title: subdomain URL matching doesn't work and other issues
Post by: Miquel 'Fire' Burns on February 22, 2007, 03:13:19 AM
Ah! That bit.

Maybe PasswordMaker should make a RegEx or something that allows it to match to what the default would be (In the above example, https?://([^/]\.)?acme\.com/.* if only domain was checked)

Actually, the default RegEx should be protocol independent as well if it wasn't selected in the default settings.
Title: subdomain URL matching doesn't work and other issues
Post by: Eric H. Jung on February 23, 2007, 02:04:32 PM
I see. Why is Adam the only person who's requested this in PasswordMaker's 2-3 history? Is this really a gap in PasswordMaker, or is Adam using it differently than other people?
Title: subdomain URL matching doesn't work and other issues
Post by: Miquel 'Fire' Burns on February 23, 2007, 02:28:40 PM
Because most of us using accounts so far had them made before the new URL pattern feature and haven't actually made a new account yet.
Title: subdomain URL matching doesn't work and other issues
Post by: Eric H. Jung on February 26, 2007, 01:27:51 AM
OK, well let's add this as a feature request... not a bug. Tanstaafl, can you add it?
Title: subdomain URL matching doesn't work and other issues
Post by: adamspiers on February 26, 2007, 09:27:10 AM
Quote from: Eric H. Jung
OK, well let's add this as a feature request... not a bug. Tanstaafl, can you add it?

Thanks.  While I'm eagerly awaiting the new version (will gladly help with beta testing, by the way), please can you explain the point of the "URLs" tab in the "Defaults" account, since I was clearly mistaken that it would affect the domain matching algorithm?
Title: subdomain URL matching doesn't work and other issues
Post by: Eric H. Jung on February 28, 2007, 11:51:51 PM
Quote from: Adam Spiers


Thanks.  While I'm eagerly awaiting the new version (will gladly help with beta testing, by the way), please can you explain the point of the "URLs" tab in the "Defaults" account, since I was clearly mistaken that it would affect the domain matching algorithm?
It's there so you can define how to use auto-populate for default accounts, which was accidentally removed from the last release.
Title: Re: subdomain URL matching doesn't work and other issues
Post by: adamspiers on May 20, 2007, 02:32:49 PM
It's there so you can define how to use auto-populate for default accounts, which was accidentally removed from the last release.


Sounds interesting - but I don't understand at all!  Why would you ever need auto-populate for default accounts?  And how are the settings for URL components supposed to affect auto-population for default accounts?  Aren't URL components used to determine how to generate a password?  Whereas auto-population as I understand it has nothing to do with password generation.
Title: Re: subdomain URL matching doesn't work and other issues
Post by: tanstaafl on May 20, 2007, 11:53:58 PM
Ok, here we go... somehow, I completely missed this entire thread... sorry about that Eric...

Now...

Sounds interesting - but I don't understand at all!  Why would you ever need auto-populate for default accounts?
?

Enabling auto-pop on the Defaults simply causes a prompt for the MPW - or, if you have saved it to memory or disk, it simply auto-pops the username/password - for any form that contains what PWM detects as a username and/or password field, *and* whose URL does not match anything in one of your Custom Accounts (so will use the 'Default' account settings).

Quote
And how are the settings for URL components supposed to affect auto-population for default accounts?
They don't - they only affect what parts of the URL are used for password calculation...
I was actually kind of confused by Erics answer too... ;)
Title: Re: subdomain URL matching doesn't work and other issues
Post by: tanstaafl on May 21, 2007, 12:03:20 AM
Eric - for purposes of this as a Feature Request...

It seems to me, after reading through this a few times, there are two parts to this...

1. Automatic URL pattern creation from the current URL when an Account is created, and

2. Automatic phishing protection.

It seems we've had a few discussions about #2... it has always been my position that the URLs that PWM uses to calculate passwords should *always* protect the user from phishing attacks.

When I add a URL as a pattern - or PWM does it for me when that feature is added - the parts of the URL that render it subject to phishing attacks should be automatically handled by PWM.

By this I mean...

If I manually add a URL right now as "*.mydomain.com", PWM should only allow VALID child URLs of mydomain.com... in other words, it should be the same as the regex Adam provided in his first post:

https?://[^/]+\.mydomain\.com/.*

Make this automatic protection optional, if desired, but I think it should be on by default, with a big fat warning if the user disables it.

I believe that satisfying both #1 (4th in the FR list with 9 votes) and #2 would completely resolve Adams concerns - and I think would be a huge improvement in the New User Experience (and old alike)... ;)

So - Adam - did I read your concerns right? Or did I miss the mark completely?
Title: Re: subdomain URL matching doesn't work and other issues
Post by: Eric H. Jung on May 25, 2007, 03:47:28 AM
Quote from: tanstaafl
If I manually add a URL right now as "*.mydomain.com", PWM should only allow VALID child URLs of mydomain.com... in other words, it should be the same as the regex Adam provided in his first post:

https?://[^/]+\.mydomain\.com/.*

OK, but which do I use as the "master" pattern; i.e., the pattern against which all future patterns are compared to see if they are children?
Title: Re: subdomain URL matching doesn't work and other issues
Post by: adamspiers on May 25, 2007, 10:24:40 AM
tanstaafl, I think you've correctly understood the functionality that I'm after, but you're making an incorrect assumption about how to implement it.  It is not necessary to automatically create a URL pattern when an account is created.  Let me explain...

First, open the 'Defaults' account and select the 'URLs' tab.  Now check and uncheck some of the URL Components buttons and observe how the 'Calculated URL of current site' changes accordingly.  This is effectively a hashing function, where the inputs are the current URL and which URL components are checked, and the output is the value displayed.  This hashing function is currently only used as one of the ingredients in the password generation algorithm, but it should also be used to determine which (if any) account matches the current URL as follows:

  1. Each time a new URL is opened, the hashing function is applied.
  2. The resulting value is compared against the 'Use the following URL to calculate the generated password' value of all the existing accounts.
  3. If only one account matches, that is the one whose settings are applied.
  4. If there are multiple matches, the menu allowing the user to choose should appear, as per normal.

Taking this approach not only eliminates the need for autocreation of regexps or other patterns, but it also protects completely against phishing attacks.
Title: Re: subdomain URL matching doesn't work and other issues
Post by: Miquel 'Fire' Burns on May 25, 2007, 01:52:46 PM
So you mean, in the case of this site, the defaults with the settings of just domain checked, it will search for accounts that have 'passwordmaker.org' as it's use URL field (plus, do the normal url stuff that's done now I would hope, just ignoring the accounts that already matched so far maybe)
Title: Re: subdomain URL matching doesn't work and other issues
Post by: tanstaafl on May 25, 2007, 03:36:56 PM
tanstaafl, I think you've correctly understood the functionality that I'm after, but you're making an incorrect assumption about how to implement it.  It is not necessary to automatically create a URL pattern when an account is created.
Not necessary, no - but highly desired... it will make creating new accounts *much* easier.

Quote
Let me explain...

First, open the 'Defaults' account and select the 'URLs' tab.  Now check and uncheck some of the URL Components buttons and observe how the 'Calculated URL of current site' changes accordingly.  This is effectively a hashing function, where the inputs are the current URL and which URL components are checked, and the output is the value displayed.
Correct so far...

Quote
This hashing function is currently only used as one of the ingredients in the password generation algorithm,
Not precisely true...

It is automatically used by the Default Account, yes - and it also provides the default *value* for the 'Use the following URL...' field when a new specific Account is created - but this value can easily be changed (and I like the fact that this can be changed).

Quote
but it should also be used to determine which (if any) account matches the current URL as follows:

  1. Each time a new URL is opened, the hashing function is applied.
  2. The resulting value is compared against the 'Use the following URL to calculate the generated password' value of all the existing accounts.
  3. If only one account matches, that is the one whose settings are applied.
  4. If there are multiple matches, the menu allowing the user to choose should appear, as per normal.
Actually, I think it is you who may be misunderstanding how PWM is intended to work... ;)

The current URL and the 'Use the following URL...' values are intended to be separate, independent and unique - and I have actually suggested in the past to change the description of the latter to something else, like 'Use the following value/string...' - precisely because it does *not* have to be a/the URL. This is convenient in that you don't have to worry about incorrect passwords being generated if/when a site makes a minor change in the URL of the login page, as long as you define the pattern(s) correctly.

If you do already understand this, then I respectfully disagree with you that it should be hard-coded to automatically and only use the calculated URL to match an account.

I wouldn't necessarily be against this as an *option*, as long as it is an option.

Quote
Taking this approach not only eliminates the need for autocreation of regexps or other patterns, but it also protects completely against phishing attacks.
As I have explained, having them separate provides many advantages, and I'd not like to see this changed, unless it is only as an option. Also, I don't use any of the other URL components other than the default 'domain', but I don't see how doing it the way you are suggesting would protect against phishing attacks.

In fact, this has been discussed on this board in the past (developing an algorithm against phishing attacks) - and, in fact, I'm replying to Erics post, which will provide more detail on this question...
Title: Re: subdomain URL matching doesn't work and other issues
Post by: tanstaafl on May 25, 2007, 11:18:17 PM
Ok...

I did a bit of searching and reading the prior threads we have had over the last year or so on phishing protection before posting this. Sorry this is so long, but I wanted to be clear and precise...

First, I think we are all agreed that for Advanced Options/Custom Account users, the current wildcard/regex capability is excellent, and more than sufficient - *but* - it is also very error-prone, even for people who are regex experts. How often have you typo'd something when doing anything on a keyboard? Well, one mistake in your regex, and you have a security hole that you aren't aware of.

Second, I think (hope) we are all agreed that it would be a 'good thing' to have a similarly effective, hard-coded phishing protection algorithm built into PWM that is used automatically for anyone using the Defaults/Basic Options, and that its just a matter of proper implementation.

See this thread (http://forums.passwordmaker.org/index.php/topic,866.30.html) for what I think is the best discussion on the issue of phishing protection - the last page of posts has the majority of the meat, though most of it is only applicable to Specific Accounts, not the Defaults.

After wracking my brain on how best to provide a similar level of protection for Defaults/Basic Options users as is available for Advanced users, I realized that, because there is nothing stored for the current URL, and all that we have to go by *is* the current URL, really, the best we can do is a small UI change of the Basic Options/MPW Prompt window (see this screenshot (http://img180.imageshack.us/img180/4084/pwmantiphishbasicoi8.jpg) of how this could look):

1. Display the 'Using URL' field on the Basic Options/Master Password Prompt window as an uneditable field (optionally making it editable by clicking a button, so that it can still be used for generating passwords on the fly and not based in any way on the current URL). Whether it is editable by default could be configurable, but I think it should not be editable by default.

2. Move the 'Using URL' field to the top of the window (above the Master Password entry field), and display it in big red letters, with a smaller explanation/recommendation in black to make sure that the domain that PWM sees is the one they intended to be accessing (ie, google.com instead of badsite.com).

3. Relabel it to 'Current Domain URL', or something more end-user friendly - not everyone knows what a TLD since it doesn't have to be a URL, and, in fact, since the user can enter anything they want here, labeling it 'Using URL' isn't always going to be appropriate.

4. Include a link for 'more info on URL/phishing scams' to a page (maybe this wikipedia article (http://en.wikipedia.org/wiki/Phishing#Phishing_techniques)) that explains in detail the dangers of phishing attacks.

Optional:
5. Instead of making the 'Using URL'/'Current Domain URL' editable by clicking a button, create a new field 'Use this text string', that if the user enters anything into it, this is used instead of the 'Current Domain URL'.

Now, I'd also like to see a simplified way of implementing the same, fundamental protection level for Advanced Options Users, with the option of enabling 'Advanced URL Regex Filtering' (or whatever it would be called), which would provide the current fine-grained, highly technical functionality.

Tomorrow I'll upload another screenshot for how the Advanced URL's tab could look to provide this, and I welcome any comments/suggestions for improvements...
Title: Re: subdomain URL matching doesn't work and other issues
Post by: adamspiers on May 26, 2007, 01:00:36 AM
Not necessary, no - but highly desired... it will make creating new accounts *much* easier.

No you misunderstand me!  I'm not disputing that automatic matching of new accounts is a very good thing.  In fact achieving this is the entire reason I started this thread in the 1st place!  What I am saying is that there is a much better way of achieving automatic matching than auto-creating a new regex each time an account is created and then matching against it.  That better way is the 4-step algorithm I just suggested.

Quote
Not precisely true...

It is automatically used by the Default Account, yes - and it also provides the default *value* for the 'Use the following URL...' field when a new specific Account is created - but this value can easily be changed (and I like the fact that this can be changed).

Yes, I realise that, although I don't see why you would want to change it.  But I think we're just about to discuss that below ...

Quote
The current URL and the 'Use the following URL...' values are intended to be separate, independent and unique

You don't really mean "current URL", here do you?  The URL of the current page is a full-blown URL, whereas the "Use the following URL" values are most likely just domain names or sub-domain names.  So they HAVE to be separate, and this is so obvious that I guess I'm missing whatever point you're trying to make.

That said, I also very much believe that there should be (conceptually) two separate, independent and unique values - one is used to generate the password, and the other to match the "resulting value" against in step 2 of my proposed algorithm.

Quote
- and I have actually suggested in the past to change the description of the latter to something else, like 'Use the following value/string...' - precisely because it does *not* have to be a/the URL.

That's an excellent idea; please do it!

Quote
This is convenient in that you don't have to worry about incorrect passwords being generated if/when a site makes a minor change in the URL of the login page, as long as you define the pattern(s) correctly.

Can you give an example of a minor change, and explain how my proposed algorithm would result in incorrect passwords being generated?  The whole point of the URL->domain hashing function is to ignore such minor changes.

Quote
If you do already understand this, then I respectfully disagree with you that it should be hard-coded to automatically and only use the calculated URL to match an account.

I wouldn't necessarily be against this as an *option*, as long as it is an option.
As I have explained, having them separate provides many advantages, and I'd not like to see this changed, unless it is only as an option. Also, I don't use any of the other URL components other than the default 'domain', but I don't see how doing it the way you are suggesting would protect against phishing attacks.

In fact, this has been discussed on this board in the past (developing an algorithm against phishing attacks) - and, in fact, I'm replying to Erics post, which will provide more detail on this question...

Sure, I'm not against making something an option.  But I would be grateful if you could reply to each of my queries above, because I think we still haven't completely aligned on what we're talking about!

I will deal with the phishing side of things in my next post in a few minutes ...
Title: Re: subdomain URL matching doesn't work and other issues
Post by: adamspiers on May 26, 2007, 01:19:22 AM
Ok...

I did a bit of searching and reading the prior threads we have had over the last year or so on phishing protection before posting this. Sorry this is so long, but I wanted to be clear and precise...

First, I think we are all agreed that for Advanced Options/Custom Account users, the current wildcard/regex capability is excellent, and more than sufficient - but - it is also very error-prone, even for people who are regex experts. How often have you typo'd something when doing anything on a keyboard? Well, one mistake in your regex, and you have a security hole that you aren't aware of.

Couldn't agree more ... so far ;)

Quote

Second, I think (hope) we are all agreed that it would be a 'good thing' to have a similarly effective, hard-coded phishing protection algorithm built into PWM that is used automatically for anyone using the Defaults/Basic Options, and that its just a matter of proper implementation.

See this thread (http://forums.passwordmaker.org/index.php/topic,866.30.html) for what I think is the best discussion on the issue of phishing protection - the last page of posts has the majority of the meat, though most of it is only applicable to Specific Accounts, not the Defaults.

Unfortunately that link doesn't work for me.  I get:

An Error Has Occurred!
The topic or board you are looking for appears to be either missing or off limits to you.

Quote
After wracking my brain on how best to provide a similar level of protection for Defaults/Basic Options users as is available for Advanced users, I realized that, because there is nothing stored for the current URL, and all that we have to go by *is* the current URL, really, the best we can do is a small UI change of the Basic Options/MPW Prompt window (see this screenshot (http://img180.imageshack.us/img180/4084/pwmantiphishbasicoi8.jpg) of how this could look):

1. Display the 'Using URL' field on the Basic Options/Master Password Prompt window as an uneditable field (optionally making it editable by clicking a button, so that it can still be used for generating passwords on the fly and not based in any way on the current URL). Whether it is editable by default could be configurable, but I think it should not be editable by default.

As I said in my previous post, I still don't understand why you would need to change this manually, but let's stick to the phishing discussion here ...

Quote
2. Move the 'Using URL' field to the top of the window (above the Master Password entry field), and display it in big red letters, with a smaller explanation/recommendation in black to make sure that the domain that PWM sees is the one they intended to be accessing (ie, google.com instead of badsite.com).

Right, now we've hit a major gap between my understanding and yours.  Please explain to me exactly how a malicious site called badsite.com would ever manage to hoodwink the extension into displaying google.com at this point?  I can only assume you are thinking of some URL like http://naughty.badsite.com/is/spoofing/google.com/innocent.html, but the URL->domain hashing function in PWM will always convert this to badsite.com, not to google.com!  I really don't see the risk here.

The only risk I see is if someone uses the Advanced Options, and configures an unsafe pattern or regular expression.  So if there is going to be a UI change anywhere to introduce a warning, it should be in that dialog box, not in the Basic Settings one.

Quote
3. Relabel it to 'Current Domain URL', or something more end-user friendly - not everyone knows what a TLD since it doesn't have to be a URL, and, in fact, since the user can enter anything they want here, labeling it 'Using URL' isn't always going to be appropriate.

As I already said, agreed 100%.

Quote
4. Include a link for 'more info on URL/phishing scams' to a page (maybe this wikipedia article (http://en.wikipedia.org/wiki/Phishing#Phishing_techniques)) that explains in detail the dangers of phishing attacks.

Again, I see zero risk of phishing in Basic Options, so a link in that dialog would be unnecessary.

Quote
Optional:
5. Instead of making the 'Using URL'/'Current Domain URL' editable by clicking a button, create a new field 'Use this text string', that if the user enters anything into it, this is used instead of the 'Current Domain URL'.

Still don't see the need for that, but if you can provide a use case I don't see a problem with it.

Quote
Now, I'd also like to see a simplified way of implementing the same, fundamental protection level for Advanced Options Users, with the option of enabling 'Advanced URL Regex Filtering' (or whatever it would be called), which would provide the current fine-grained, highly technical functionality.

Tomorrow I'll upload another screenshot for how the Advanced URL's tab could look to provide this, and I welcome any comments/suggestions for improvements...

Well, I already made two suggestions about this.  The first is my 4-step algorithm which in the vast majority of cases eliminates the need for any pattern entry (and thereby any risk of the user writing unsafe patterns).  The second suggestion which I made in another thread a long time ago is an extension to the non-regex pattern syntax:

Here's a simple user-friendly solution for the security risk people have been worrying about in this thread: in wildcard mode, stop a normal * glob from matching forward slashes, and also have a new glob syntax ** which does match them.  Then you can safely recommend users to use wildcards like:

  *://*.yahoo.com/**

Title: Re: subdomain URL matching doesn't work and other issues
Post by: tanstaafl on May 26, 2007, 05:38:47 PM
Unfortunately that link doesn't work for me.  I get:

An Error Has Occurred!
The topic or board you are looking for appears to be either missing or off limits to you.
Hmmm... that discussion is in the 'Design and Development' forum... sorry about that.

With Erics permission, I'll move it to the 'Feature Request' forum so you can read it...

Eric?

Quote
Please explain to me exactly how a malicious site called badsite.com would ever manage to hoodwink the extension into displaying google.com at this point?  I can only assume you are thinking of some URL like http://naughty.badsite.com/is/spoofing/google.com/innocent.html, but the URL->domain hashing function in PWM will always convert this to badsite.com, not to google.com!  I really don't see the risk here.
Hmmm.. you know, I think you're right... and something is very familiar about this conversation... I think I've been corrected on this before by Eric or someone else... ;)

Quote
The only risk I see is if someone uses the Advanced Options, and configures an unsafe pattern or regular expression.  So if there is going to be a UI change anywhere to introduce a warning, it should be in that dialog box, not in the Basic Settings one.
Ok, you convinced me... :)

Quote
Well, I already made two suggestions about this.  The first is my 4-step algorithm which in the vast majority of cases eliminates the need for any pattern entry (and thereby any risk of the user writing unsafe patterns).  The second suggestion which I made in another thread a long time ago is an extension to the non-regex pattern syntax:

Quote
Here's a simple user-friendly solution for the security risk people have been worrying about in this thread: in wildcard mode, stop a normal * glob from matching forward slashes, and also have a new glob syntax ** which does match them.  Then you can safely recommend users to use wildcards like:

  *://*.yahoo.com/**
Actually, I can think of something even simpler...

Just add a little checkbox that says something like 'Enable Built-in Anti-Phishing Protection' (with a 'More info' link that will provide detailed information on the algorithm used and how and when it is applied) that is enabled by default for new accounts (but disabled on existing accounts for backwards compatability), that applies the following algorithm to a basic TLD entry like 'yahoo.com':

Only match a properly formatted preceding "https?://*.?" (I think I did that right) with no embedded redirects (or whatever you call them) in the path... so, just putting 'yahoo.com' would all by itself accomplish the same thing as your example - unless I missed something again...

Then, the same algorithm could also be applied to custom regex and wildcard entries (even existing ones), but only in a 'Warning' mode - ie, when you open an existing account and look at the URLs Tab, if any of the existing wildcard/regex entries are unsafe, they would be flagged somehow (but would continue to work as they did previously). This would enable anyone to fix existing broken ones when convenient for them. This would also allow a visual indicator when creating new ones to warn of any typos or unsafe expressions...

Lastly, when the above Checkbox is enabled, the Custom Regex/Wildcard list/management stuff is hidden - when unchecked, it is visible...
Title: Re: subdomain URL matching doesn't work and other issues
Post by: tanstaafl on May 26, 2007, 05:46:31 PM
Quote
Actually, I can think of something even simpler...
And, of course, I posted that *before* I went back and re-read your 4-step algorithm...

I think that your 4-step algorithm is actually accomplishing the same thing, *except* that it depends on the user actually checking the all of the different URL parts.

My method doesn't - it *automatically* applies them as I described - then just compares them to the TLD of the account.

One thing I missed - the 'Use the following URL...' field would need to be changed to something like 'URL TLD for this Account'. The main issue I see is this field should not be easily editable, because if the user changes it, there would not be a match.
Title: Re: subdomain URL matching doesn't work and other issues
Post by: adamspiers on May 26, 2007, 09:03:20 PM
Firstly, thanks for not giving up on the discussion yet ;)  It's proving a very difficult topic to discuss in this communication medium, but I think we're gradually making progress...

Hmmm... that discussion is in the 'Design and Development' forum... sorry about that.

With Erics permission, I'll move it to the 'Feature Request' forum so you can read it...

Great, thanks.

Quote
Ok, you convinced me... :)

;)

Quote
Actually, I can think of something even simpler...

Just add a little checkbox that says something like 'Enable Built-in Anti-Phishing Protection' (with a 'More info' link that will provide detailed information on the algorithm used and how and when it is applied) that is enabled by default for new accounts (but disabled on existing accounts for backwards compatability), that applies the following algorithm to a basic TLD entry like 'yahoo.com':

Only match a properly formatted preceding "https?://*.?" (I think I did that right) with no embedded redirects (or whatever you call them) in the path... so, just putting 'yahoo.com' would all by itself accomplish the same thing as your example - unless I missed something again...

Then, the same algorithm could also be applied to custom regex and wildcard entries (even existing ones), but only in a 'Warning' mode - ie, when you open an existing account and look at the URLs Tab, if any of the existing wildcard/regex entries are unsafe, they would be flagged somehow (but would continue to work as they did previously). This would enable anyone to fix existing broken ones when convenient for them. This would also allow a visual indicator when creating new ones to warn of any typos or unsafe expressions...

Lastly, when the above Checkbox is enabled, the Custom Regex/Wildcard list/management stuff is hidden - when unchecked, it is visible...

I'm struggling to understand the above, sorry - but it sounds like you're still suggesting taking a domain like 'yahoo.com', converting it into a regexp like 'https?://[^/]+\.yahoo\.com/.*', and then matching the URL of the current page against that regexp.  But in my opinion this is unnecessarily complicated and the wrong way round.  Why?  Well, the end goal is to find out whether the URL of the current page corresponds to a particular account, right?  So rather than converting the domain of a particular account to something more complicated (a regexp) and then doing a regexp match, it's much easier to convert the current URL to something simpler (a domain) and then do a simple string equality test.  And as I said before, this latter conversion is a hashing function which is controlled by the components checkboxes in the Defaults account.

Secondly, regexps are much more complicated to work with than you might think.  Analysing whether a particular regexp is subject to phishing attacks is distinctly non-trivial, so I would strongly suggest the following approach:

  (1) Greatly reduce the need for any user (even an advanced one) to write regexps
  (2) If a user really knows what they are doing and really needs a regexp, then they should be allowed enough rope to hang themself, with appropriate warnings about the risks of phishing.

The first (1) is solved in two ways.  Firstly, applying the hashing function to the current URL and comparing it against the automatically generated "Use the following..." value for each account, as described in my 4-step algorithm, eliminates the need for regexps in 90% of cases. 

In the other 10%, the user wants a single account to match multiple domains (say).  So in addition to the "Use the following..." value which will be used for matching against the domain the account was originally created for, the user needs to be able to specify extra domains which the account will also match.

For instance, let's assume that microsoft.com and msn.com are federated into a single sign-on system, which for the sake of example requires passwords which only contain letters and digits.  So the user creates a new account 'microsoft.com' which auto-generates passwords satisfying this constraint.  Automatically, the 4-step algorithm ensures that the same password is generated for http://login1.microsoft.com/foo/bar and https://login2.microsoft.com/baz/qux.  Now the user wants to make this same account match for the msn.com domain.  This is where the second solution for the 10% of cases comes in.  The user simply adds a new pattern *://*.msn.com/** to the account, which is phishing-proof as per my ** syntax extension suggestion, or if they want, they add a new regexp https?://[^/]\.msn\.com/.* which achieves almost the same thing.

Or, even better, we could extend/change the UI so that in addition to being able to specify glob and regular expression patterns, the user can specify additional domains to do the plain string equality test against - in other words, they are treated identically to the 'Use the following...' value.  IMHO this is the safest option and simplest for the user to understand.

The second (2) is solved simply by adding a warning to the UI in the place where new patterns can be added.

And, of course, I posted that *before* I went back and re-read your 4-step algorithm...

I think that your 4-step algorithm is actually accomplishing the same thing, *except* that it depends on the user actually checking the all of the different URL parts.

I don't understand - which URL parts do you mean?  I don't see how it depends on the user checking anything.

Quote
My method doesn't - it *automatically* applies them as I described - then just compares them to the TLD of the account.

Applies what?  Are you talking about that TLD domain->regexp conversion again?  If so, recall that I suggest above that IMHO it's unnecessary and over-complicated.

Quote
One thing I missed - the 'Use the following URL...' cox would need to be changed to something like 'URL TLD for this Account'. The main issue I see is this field should not be easily editable, because if the user changes it, there would not be a match.

Agreed.  Although 'URL TLD' is not correct either, because the user might check the subdomain checkbox in the URL components bit of the Defaults account.  In that case, login1.msn.com and login2.msn.com would be deliberately treated differently.
Title: Re: subdomain URL matching doesn't work and other issues
Post by: adamspiers on April 12, 2009, 02:27:51 PM
Hi all,

It's almost 2 years since I originally raised these usability issues, and nothing seems to have changed.  It's still a very cumbersome process whenever I encounter a new website which has a stupid restriction on how it accepts passwords (e.g. alphanumeric only, or not longer than 7 characters):

  1. create a new custom account (or copy from an existing one)
  2. go to the URLs tab
  3. click Add Pattern
  4. enter a Pattern Name
  5. enter something like https?://[^/]+\.new\.annoying\.site\.com/.*
  6. select the 'Regular Expressions' checkbox
  7. submit
  8. hope I got the regexp right (it used to be possible to check this by hovering over the ring icon but that doesn't work anymore)
  9. hope I didn't open myself up to a phishing attack by getting the regexp slightly wrong

This is surely a very common use case for any passwordMaker user, and yet it's so complicated to do that I expect hardly anyone does it.

Is passwordmaker development still alive?

Thanks,
Adam
Title: Re: subdomain URL matching doesn't work and other issues
Post by: tanstaafl on April 12, 2009, 03:50:29 PM
We're all still here - yes, Eric (the creator and main developer for PWM) too - but there haven't been any new releases in a long time...

I've been kind of hoping that Eric has been working in secret on some of the Feature Requests that would make such a huge difference in both the learning curve and the ease of use, like the Simplified 'Basic' and 'Advanced' GUI changes (http://forums.passwordmaker.org/index.php/topic,1385.msg1280493.html#msg1280493) (which goes to this post/complaint of yours specifically), and the New 'Default' & 'Advanced' Security Modes (http://forums.passwordmaker.org/index.php/topic,1359.msg1279932.html#msg1279932) - which resulted from a discussion between you and me, and I never heard back from you on.

Of course there are others that will also make using PWM much easier and more eficient...

Ability to sort accounts tree by column (http://forums.passwordmaker.org/index.php?showtopic=196)

Ability to create a new account for the current URL via context menu while in a password field (http://forums.passwordmaker.org/index.php?topic=566)

Modifiable Username Prompt et al (http://forums.passwordmaker.org/index.php?showtopic=256)

Multiple bUsernames per account, by pattern (http://forums.passwordmaker.org/index.php/topic,1420.msg1280840.html#msg1280840)

But imo, the first two dealing with simplifying the GUI and usage are most important...
Title: Re: subdomain URL matching doesn't work and other issues
Post by: Eric H. Jung on April 13, 2009, 04:29:20 AM
Adam,

not sure what to say except that contributions by people like yourself are more than welcome. i bet you could have mastered passwordmaker internals in 2 years, so where's the patch?  :)

Eric
Title: Re: subdomain URL matching doesn't work and other issues
Post by: adamspiers on April 13, 2009, 10:54:25 AM
We're all still here - yes, Eric (the creator and main developer for PWM) too - but there haven't been any new releases in a long time...

I've been kind of hoping that Eric has been working in secret on some of the Feature Requests that would make such a huge difference in both the learning curve and the ease of use, like the Simplified 'Basic' and 'Advanced' GUI changes (http://forums.passwordmaker.org/index.php/topic,1385.msg1280493.html#msg1280493) (which goes to this post/complaint of yours specifically), and the New 'Default' & 'Advanced' Security Modes (http://forums.passwordmaker.org/index.php/topic,1359.msg1279932.html#msg1279932) - which resulted from a discussion between you and me, and I never heard back from you on.

You're right, I do owe you a final reply to that thread - I'll try to do that today.

Quote
Of course there are others that will also make using PWM much easier and more eficient...

Ability to sort accounts tree by column (http://forums.passwordmaker.org/index.php?showtopic=196)

Ability to create a new account for the current URL via context menu while in a password field (http://forums.passwordmaker.org/index.php?topic=566)

Modifiable Username Prompt et al (http://forums.passwordmaker.org/index.php?showtopic=256)

Multiple bUsernames per account, by pattern (http://forums.passwordmaker.org/index.php/topic,1420.msg1280840.html#msg1280840)

But imo, the first two dealing with simplifying the GUI and usage are most important...

Agreed, that all sounds good.
Title: Re: subdomain URL matching doesn't work and other issues
Post by: adamspiers on April 13, 2009, 10:59:40 AM
Adam,

not sure what to say except that contributions by people like yourself are more than welcome. i bet you could have mastered passwordmaker internals in 2 years, so where's the patch?  :)

Eric


Maybe I could if I had the time, but I didn't.  I did look at the source code a couple of times but it didn't look like it was going to be a simple task given my less-than-fluent Javascript and non-existent XUL.
Title: Re: subdomain URL matching doesn't work and other issues
Post by: Eric H. Jung on April 13, 2009, 02:31:34 PM
Maybe I could if I had the time, but I didn't [have the time]

Ditto.