• Esp@lemmy.blahaj.zone
      link
      fedilink
      English
      arrow-up
      25
      ·
      1 year ago

      Yeah, this same exact story keeps coming up for years now just with different names. Why anyone would think that both the ineffectiveness and racial bias in these systems either wouldn’t exist or will somehow go away eventually is beyond me. Just expensive and ineffective mass surveillance for the sake of it…

    • steltek@lemm.ee
      link
      fedilink
      English
      arrow-up
      19
      ·
      edit-2
      1 year ago

      Who remembers the HP computer that was unable to identify black people? One of my favorite “oooph, that’s not a good look” tech fails of all time. At least the people in that video were having a good laugh about it.

      https://www.youtube.com/watch?v=t4DT3tQqgRM

      Holy hell, that was 13 years ago.

    • oct2pus@kbin.social
      link
      fedilink
      arrow-up
      10
      ·
      1 year ago

      Minor correction.
      15 out of 15 requests were of black people. 14 of those requests were black men and 1 was a black woman.

    • bob_wiley@lemmy.world
      link
      fedilink
      English
      arrow-up
      7
      arrow-down
      1
      ·
      1 year ago

      While true, it is important to remember that New Orleans is a predominately black city, with ~58% of the population, compared to ~12% nationally. So it does make sense there would be more requests of black people than you may typically expect. Maybe not 14/15 high, but 9 out of 15 would be expected (if we assume all races commit crimes at the same rate).

      https://www.census.gov/quickfacts/fact/table/neworleanscitylouisiana/PST045222

      • yA3xAKQMbq@lemm.ee
        link
        fedilink
        English
        arrow-up
        17
        ·
        edit-2
        1 year ago

        Yeah, but statistics is a b*tch.

        We had a similar technology for a test run some years ago at a train station in Berlin, capital of Germany and largest city in the EU with 3.8M.

        The results the government happily touted as a success were devastating. They had a true positive rate of 80% (and this was already cooked since they tested several systems at several locations but only reported the best results), which is really not that good to start with.

        But they were also extremely proud of the false negative positive rate, which was below 0.1%. That doesn’t sound too bad, does it?

        Well, let’s see…

        True positive means you actually identified the people you were looking for. Now, I don’t know the number of people Berlin’s police is actively looking for, but it’s not that much. And the chances of one of them actually passing that very station are even worse. And out of that, you have 20% undetected. That’s one out of five. Great. If I were a terrorist, I would happily take that chance.

        So now let’s have a look at the false negative positive rate, which means you incorrectly identified a totally harmless person as a terrorist/infected/whatever. The population for that condition is: everyone passing through that station.

        Let’s assume there’s a 100k people on any given day (which IIRC is roughly half of what that station in Berlin actually has). 0.1% of 100k is 100 people, every day, who are mistakenly reported as „terrorists“. Yay.

    • Blamemeta@lemmy.world
      link
      fedilink
      English
      arrow-up
      4
      arrow-down
      2
      ·
      edit-2
      1 year ago

      Yeah. Basicly anything with a lower contrast, with shadows and backgrounds. And because shadows are dark, they have a lower contrast with other dark things.

    • P03 Locke@lemmy.dbzer0.com
      link
      fedilink
      English
      arrow-up
      4
      arrow-down
      9
      ·
      1 year ago

      Discrimination is the wrong word. Technology has no morals or sense of justice. It is bias in the data that developers should have accounted for.

      • steltek@lemm.ee
        link
        fedilink
        English
        arrow-up
        11
        ·
        1 year ago

        It’s totally accurate though. It’s like the definition of systemic racism really. Think about housing or financial policy that disproportionately fails for minorities. They aren’t some Klan manifesto. Instead they just include banal qualifications and exemptions that end up at the same result.

      • Cortell@lemmy.world
        link
        fedilink
        English
        arrow-up
        7
        ·
        1 year ago

        Ask the people who create the data sets that machine learning models train on how they feel about racism and get back to us

      • Corhen@lemmy.world
        link
        fedilink
        English
        arrow-up
        6
        ·
        1 year ago

        It can be an imported bias/descrimination. I still think that words fair.

        Do you have a more accurate word?

        • P03 Locke@lemmy.dbzer0.com
          link
          fedilink
          English
          arrow-up
          2
          ·
          1 year ago

          I already said it: bias. It’s a common problem with LLMs and other machine learning models that model engineers need to watch out for.

      • slumberlust@lemmy.world
        link
        fedilink
        English
        arrow-up
        6
        ·
        1 year ago

        This seems shortsighted. You are basically asking people to police their own biases. That’s a tall ask for something no one can claim immunity from.

        • P03 Locke@lemmy.dbzer0.com
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 year ago

          I am asking a group of scientists who should be very well-versed in statistics and weights, you know, one of the biggest components in a machine learning model, to account for how biased their data is when engineering their model.

          It’s really not a hard ask.

      • HardlightCereal@lemmy.world
        link
        fedilink
        English
        arrow-up
        9
        arrow-down
        11
        ·
        1 year ago

        You need to learn some critical race theory. Racist systems turn innocent intentions into racist actions. If a PhD student trains an AI model on only white people because the university only has white students, then that AI model is going to fail black people because black people were already failed by university admissions. Innocent intention plus racist system equals racist action.

  • Blackmist@feddit.uk
    link
    fedilink
    English
    arrow-up
    46
    arrow-down
    4
    ·
    1 year ago

    I’m going to take a wild stab in the dark that all the false positives were black men.

    For the same reason that my Echo dot (aka Spotify Bitch) will ignore my wife but cheerfully respond to my mumbled requests from three rooms away. If you make all this shit in Silicon Valley, it will work best for people of a similar demographic to those that work there.

    • soviettaters@lemmy.world
      link
      fedilink
      English
      arrow-up
      20
      arrow-down
      11
      ·
      1 year ago

      The white liberals building this technology say they’re all progressive yet only surround themselves with people like them and only build products for people like them. A lack of diversity in tech like this is a lack of good testing.

      • Blackmist@feddit.uk
        link
        fedilink
        English
        arrow-up
        18
        arrow-down
        1
        ·
        1 year ago

        Oh they are progressive. They’ll support Black Lives Matter and sympathise with Iranian women.

        But there’s only so much anybody can do when it’s the entire US (and further afield) social structure at fault. It’s the same where I am. I work on a project with 3 other white guys. If I put a job advert up for another programmer, who will apply? 3-4 more white guys.

        I agree that it’s a lack of good testing. Especially when you consider that it’ll be mostly used to pick black guys out of a database. And especially so in New Orleans.

        • Puzzle_Sluts_4Ever@lemmy.world
          link
          fedilink
          English
          arrow-up
          6
          arrow-down
          1
          ·
          1 year ago

          Pretty much any tech company that is large enough to have 5 or 6 people on a team should be looking toward paid internships. It is a way to get labor for dirt cheap, help students get their five years of work experience for an entry level position, and do extended interviews for new hires.

          And the key with that is to not just advertise at your alma mater. Go out of your way to advertise to women in engineering groups and black friendly mailing lists and so forth. It gives you a MUCH better applicant pool and, if you are genuinely hiring/selecting “the best of the best”, you naturally become more diverse (because people are people).

          Which has knock on effects back at the universities because professors are very much selecting grad students based on how likely they are to get a summer internship (and thus cost much less per year). If they know that only their white male students will be able to get an internship? They get more white males. If they suddenly realize ethnicity and gender don’t matter? They too begin selecting the best of the best which, again, naturally gets diversity closer to the national averages rather than the “STEM average”.

      • OhNoMoreLemmy@lemmy.ml
        link
        fedilink
        English
        arrow-up
        11
        arrow-down
        2
        ·
        1 year ago

        They’re more libertarian than liberal. Anti worker rights, anti consumer rights, and anti taxation.

        The only government spending they’re in favour of is government spending and subsidies on tech e.g. Tesla, space X, and the entire military complex.

        • RaoulDook@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          arrow-down
          13
          ·
          1 year ago

          You haven’t read much about Libertarian policy I see. They are very pro-rights, in fact that is the core of the party platform. Individual liberty is their chief concern, and I applaud their efforts in fighting for our rights and freedom.

      • Smoogs@lemmy.world
        link
        fedilink
        English
        arrow-up
        4
        arrow-down
        6
        ·
        1 year ago

        Also AI is taught by its creator. Tech has some of it’s most well hidden, bigotted, mid-level white people refusing to critically question their own bias and privilege. There’s a shit tone of that fragile masculinity in the tech industry just hard coding it into it.

        There was a guy fired from google for writing a manifesto about how women aren’t ‘wired’ for tech. And that’s just the one that waved his crazy flag out in the open so no one in upper management could easily keep on ignoring it.

        • BrotherCod@kbin.social
          link
          fedilink
          arrow-up
          5
          ·
          1 year ago

          While I agree with you 100% that programming can be affected by the programmers biases, there’s a much simpler problem that face recognition was having a hard time overcoming. At least when it was a main topic about a decade ago, sensors were having a lot of problems with the low contrast of some black people’s faces. Anyone who’s had a black friend and was a shutter bug will know what kind of problems you can run into when trying to get a proper exposure and not make a black person disappear completely from a photograph. It was just an inherent limitation of the technology they were using. The last statistics I read was something like between 20 to 30% positive matches, which we know damn well is too low for it to be a workable technology. The success rate on Caucasian and lighter skin tones weren’t even that great. There was still something like a 60% false positive match rate. The software may have gotten better over the past decade but we all know that whether it did or not, they’re still going to use it.

          • Smoogs@lemmy.world
            link
            fedilink
            English
            arrow-up
            1
            arrow-down
            3
            ·
            1 year ago

            This isn’t image manipulation of the 1990s. You assume it’s set on isolated pixels with massive contrast. It’s calculated by neighbor to achieve the pattern.

            This is just a result of inconsideration driving laziness that they’d crop to a median level of the graphic to cater to the skin with less reflection and reads light easier and then releasing it as ‘done’. Software is much more sophisticated than you’re giving credit. But It’s only being used to that potential in such industry as film.

    • Flying Squid@lemmy.world
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 year ago

      I imagine anyone with more than a mild accent doesn’t even bother using Alexa or whatever except in their native language.

  • agitatedpotato@lemmy.world
    link
    fedilink
    English
    arrow-up
    39
    arrow-down
    2
    ·
    edit-2
    1 year ago

    The terrifying part to me is that cops across the nation have a long history of seeing that the tech they want to use is unreliable and based on junky science, but they still push it through anyway. Aren’t police dogs about as reliable as a coin-flip when their handlers aren’t nipping at their neck to get them to jump at anything? They don’t care if it’s right as long as they can use it to justify their behavior, so they make it policy.

    • yeather@lemmy.ca
      link
      fedilink
      English
      arrow-up
      27
      ·
      1 year ago

      Only the drug dogs are ineffective. Bloodhounds and tracking dogs have been a staple of hunting down people, and German retrievers can take a man down effectively as well.

      • brianorca@lemmy.world
        link
        fedilink
        English
        arrow-up
        2
        ·
        1 year ago

        When they are trained with incentives for finding something, instead of incentives to be correct, then they will find something. Same is true for man or beast.

    • Diplomjodler@feddit.de
      link
      fedilink
      English
      arrow-up
      11
      ·
      1 year ago

      A lot if forensic “science” is utter bunk. Yet it continues to be used. Having a fair and equitable system was never the point.

  • MiddleWeigh@lemmy.world
    link
    fedilink
    English
    arrow-up
    29
    ·
    1 year ago

    The current state of policing doesn’t deserve to have access to this kinda shit. Hopefully it never will tbh.

  • MisterFrog@lemmy.world
    link
    fedilink
    English
    arrow-up
    27
    ·
    1 year ago

    People may see this as a “see, AI isn’t that good”. We all need to rail against these kinds of programs to the point they are made illegal. Because there are examples around the world of being able to track people with facial recognition (and even by the way someone walks with their face entirely covered 0_0)

    I see this as the new Orleans police dep hired a inept contractor (or did an inept job in house).

    Around the world, we must fight against all inappropriate data harvesting.

    • Misconduct@lemmy.world
      link
      fedilink
      English
      arrow-up
      15
      arrow-down
      1
      ·
      edit-2
      1 year ago

      With all the laws trying to put women into basically servitude I’m definitely on team rail against. There are a lot of types of “criminals” that need to be able to get away from law enforcement these days unfortunately. Honestly I’d prefer they just keep being inept for now lol

  • quicksand@lemmy.world
    link
    fedilink
    English
    arrow-up
    16
    ·
    1 year ago

    When I walk into the building I work at there is a disclaimer that they are using facial recognition. I don’t know if this is reality or a scare tactic, but based on the industry I would assume they’re just using it for free AI training

  • SangriaFerret@lemmy.world
    link
    fedilink
    English
    arrow-up
    19
    arrow-down
    4
    ·
    edit-2
    1 year ago

    Tbf, NOPD don’t arrest many people anyway. There’s a massive cop shortage, only 944 officers for a city of 364,000 with skyrocketing crime rates. Moreover, they’ve been operating under a consent decree by the DOJ since 2012. They’re overworked, underpaid and under the thumb of the feds so in response they simply don’t do shit.

    • TurtleJoe@lemmy.world
      link
      fedilink
      English
      arrow-up
      12
      ·
      1 year ago

      The cops in my city were under a DOJ consent decree for like 20 years, and it didn’t make them any less effective. They’re actually worse now, because they actively don’t give a fuck.

    • karlthemailman@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      10
      arrow-down
      1
      ·
      1 year ago

      Serious question, what is the right number of officers for a city that size? 1 officer per 400 people or so doesn’t sound very low to me.

      • SangriaFerret@lemmy.world
        link
        fedilink
        English
        arrow-up
        6
        ·
        1 year ago

        NOPD’s stated goal is 1600, a ratio of 1:227 persons.

        The actual ratio is 1:385

        Cleveland, similar in size to New Orleans, has a ratio 1:310. They also state that they are suffering from a serious police shortage.

        By comparison:

        NYC has a ratio of 1:166

        Chicago 1:180

        • Corhen@lemmy.world
          link
          fedilink
          English
          arrow-up
          7
          ·
          1 year ago

          1:100 seems insane. I live in a community of 10,000 people, and we don’t have anything close to 100 people working in the RCMP. MAYBE 20-30, including support

          • mski@lemmy.ca
            link
            fedilink
            English
            arrow-up
            8
            ·
            1 year ago

            Just checked our major urban centre in Canada, and it’s around 1:450. As a comparison, that makes New Orleans (1:385) pretty well staffed.

            Would be cool to find data covering major urban centres across the world for comparisons.

              • Machinist3359@kbin.social
                link
                fedilink
                arrow-up
                3
                ·
                1 year ago

                And NYC is wildly over-policed.

                I swear, some people have never met a societal problem they didn’t want to throw a cop at. Meanwhile we have more cops and prisoners per Capita than most of the world, funny how that works…

  • Uriel238 [all pronouns]@lemmy.blahaj.zone
    link
    fedilink
    English
    arrow-up
    14
    ·
    1 year ago

    Well, I could have told you this. (Techdirt has plenty of articles on how facial recognition software mostly generates false positives and ruins the days, if not the lives, of innocents).

    On a similar note, the massive camera array of London, to which law-enforcement and state security departments are plugged in, is useful for less than 0.1% of incidents.

  • bob_wiley@lemmy.world
    link
    fedilink
    English
    arrow-up
    14
    arrow-down
    1
    ·
    1 year ago

    A perfect example of why it’s not worth giving up privacy in the same of “safety”.

  • MisterEspinacas@lemmy.world
    link
    fedilink
    English
    arrow-up
    8
    ·
    1 year ago

    I mean, law enforcement occasionally uses polygraph tests in their investigations even though that type of “evidence” isn’t admissible in court and, to be honest, what kind of scientific credibility does a piece of technology like a polygraph even have? They’ll use whatever they can get their hands on even if it’s questionable. Some police forces probably even have a psychic consultant or something. It scares me.

    • Soggy@lemmy.world
      link
      fedilink
      English
      arrow-up
      6
      ·
      1 year ago

      They’ll use it especially if it’s questionable, like handwriting analysis, because the goal is arrests not correct arrests. Trumped up, flimsy, circumstantial “evidence” is the best kind when you don’t actually want to do your job.

      • MisterEspinacas@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 year ago

        Yeah, it goes along with the low standards that define probable cause. Policing, just like a lot of professions, is subject to bean counting when bean counting is not appropriate. Voters love to see statistics that flaunt “more arrests.” Funny how people love numbers without really understanding what the numbers mean.

  • P03 Locke@lemmy.dbzer0.com
    link
    fedilink
    English
    arrow-up
    8
    arrow-down
    1
    ·
    edit-2
    1 year ago

    So, why not just write-off the technology as unreliable and move on? Even with the atrocious false positive rate, you would have still expected more than 15 hits in 9 months. This tech has got to be expensive and even the potential ROI on this, if it ever works at all, is very not worth it.