of the data. and for a visualization/computing project, the data format, and structures that are very important, and i, and there are a number of sort of technical issues that are, that i can touch on i guess. so one of the first things that we tried to do with the data, is simply draw a correlation between things and so you saw ash's visualization, that drew a correlation between lobbying event, and contributions. and we were interested in doing the same thing for lobbying events and voting records. and you know, and for every variable in the data set there is a corresponding correlation that can be made against all of the other variables in the data set. and so we were also interested in connecting this data set up with other ones and so beyond the 460, what can we connect it to. and so as sort of in the computing space, the term for what we are trying to accomplish is called a mash up. and so it basically a combination of multiple different data sources. off not, you know, the sources are... th

the easiest way to identify wage gaps is by collecting aggregate data, data in the aggregate on all employees and ewing categories that employer are used to reporting. so that eeo1 federal categories, and that's the easiest way to both prevent employers from being able to fudge the data in terms of presenting compensation and job classification, et cetera, to the city, and it's the easiest -- it's least burdensome for employers and it's the aloiest way to identify the gap. after talking to her i realized, wow, this is really complicated, as you mentioned, commissioner dooley. collecting this data and all of the, you know, all of the issues that go into determining a workers' compensation, years of experience, education, job title, job classification, et cetera, there are many variables. and that in order to do this right we didn't want to legislate exactly what data needs to be collected because we didn't feel that our office, or even the city attorney, or even the board of supervisors had all of that expert knowledge to say exactly how to collect data in the way that was going to b

who can see the data and what they can do with the data. the other point is oversight pecknisms. and this is really you think a lot about audit logging and also using technological electronic work flows to control exactly how data flows around an organization and who can see data and what kind of analysis they can do with it. or hard wiring an approval chain for use of data and things like that. and these can be very detailed. so the -- or the hard wired approval process and things like that. that can be very complex and involve multiple actors. and then the auditing of how data used it self can be incredibly granular and incredibly detailed. and i want to get to other point. just these two capabilities are a significant improvement of what existed before and can get us a long way. and there are things that exist today. now i'm obligated to say that poll intier does this best but this is not exclusive to poll intier and they can be deployed and can be used in a lot of different context. so what is the problem today? why aren't these capabilities being used more? a couple things.

lawful sl more is what the type of data, how intrusive is the data? how broad is the collection? in other words, am i obtaining a lot of people who are sort of an accidently collection or not part of the target. and what are the stated uses or future uses. we've been using those three questions. to get an overall risk is we to want start the government from doing bad things to good people. and so, you sort of looking through those different lenses, it helps us do that analysis. >> so thank you. david -- >> i'm sorry. >> just because you wanted -- >> i know. i was just going to sort of follow up on the comments in that. that i think forcing mechanism of trying to do, of having ongoing vetting and ongoing evaluation by the right people is where to go because you're looking for sort of, you know, the meaningful relationships and developing those as opposed to, you know, retaining the isolated pieces. seen i would just say that trying to force that mechanism of vetting is really important. >> one of the reasons for having the forum today is to get a under

the second stage of the pipeline merges the data. if two data files can be determined to correspond to the same person before example because they contain the same unique identifier, those files can be merged. merging can create an avalanche effect because merged files can be a more precise information about identity and behavior and that precision in turn allows for the emerging. one file might contain detailed information about behavior and another might contain precise identity information. version those files links behavior and identity together. the third stage of the pipeline uses big data methods such as predictive analytics to infer facts about people. one famous example is when the retailer targets products such as skin motion to infer pregnancy. today the machine learning methods often enable sensitive information to be inferred from seemingly less sensitive data. differences also have another latch effect because each inference becomes a data point to be used in making further inferences. predictive analytics are most effe

the government can answer those questions and looking at how the data is used and using the data about data, to me that is where you would expand that x beck -- reasonable expectation of privacy. >> the question you pose is a very difficult one. courts wrestle with it all the time. everyone has a personal opinion and so to live. i believe that the reasonable expectation of privacy is a terrible framework. it is supposed to protect against searches and seizures. it has now allowed for some very unreasonable searches and seizures. reasonable expectation of , u.s. versuswork jones notwithstanding, the supreme court seems to be inching along in a direction where they are doubting the reasonable expectation of privacy framework as it has been applied to the past several decades, but under current law, would it be ok to have a network of drones or ground-based cameras that watch everything you do the moment you step outside of your house. there is very strong argument that yes that is ok under the reasonable expedition of privacy framework. is the wrong framework to be doing a lot of that st

demographic data electoral data and crunch that on the backend. once we do that we are able to look at things like what is the profile of this individual legislator and how has he or she voted in the last 20 years so there happens to be a bill coming up that relates to something they have looked at before so ken bee back to the data and extrapolate for the future how to develop those outcomes. husband this is all done via an algorithm that you developed? >> guest: yes. >> host: how old are you? >> guest: i'm 22. >> host: how did you get interested in all this? >> guest: so my own career has taken a lot of twists and turns in the last couple of years but i really started off initially in the obama campaign in 2008 working initially in the field and moving to the data side initially looking at some of the demographic type issues and those types of things but moving more into the issue base things. after the campaign event i was thinking about, i was 17 at the

the second stage of the pipeline merges the data. if two data files can be determined to correspond to the same person, for example, because they both contain the same unique identifyr, then those files can be merged. and merging can create an avalanche of that, because merged files contain more precise information about identity and behavior, and that precision in turn allows further merging. one file might contain detailed information about behavior and another might contain precise identity information. merging those files links behavior and identity together. the third stage of the pipeline uses data methods such as an t analytics. today's learning methods often enable sensitive information to be lifted from seemingly insensitive data. it can have an avalanche effect because each inference becomes another data point to be used in making further inferences. data is effective in status. for instance, target used many examples of pregnant and non-pregnant women to build its predictive model. a predictive model that tried to identify

. >> the data from 2009 is the last data that has been cited in official publications by the national traffic safety commission. based ontistics are data collected annually and presently are available through 2012. 1.6%ost recent show that of drivers involved in fatal crashes in 2012 were believed to be drowsy or fatigued at the time of the crash. >> what would you say is the general trend? is there a trend, and if so, what direction is it heading? >> in the official statistics there does not seem to be much of a general trend. proportion ande the official statistics may have .ecreased slightly however, i think that is such a vast underestimate, i would not ink to the data published those sources as evidence of a trend or a lack thereof. in the evidence, this is based on the sample of about 5000 crashes each year that were investigated in depth. there was not evidence of an increasing or decreasing trend in the proportion of crashes that involved drowsiness. i would note in recent years the with injurieshes and deaths nationwide has increased, and that is a good thing. believe there't

who can see the data and what they can do with the data. the other point is oversight mechanisms, and this it really thinking a lot about audit logs and also using technological electronics work flows to control exactly how data flows around an organization and who can see data and what kind of analysis they can do with it. even awed mating or at least hardwiring in and approval chain for use of data and things like that, and these can be very detailed. so the -- or the hard wired approval process and things like that. that can be very complex and can involve multiple actors, and can involve multiple stakeholders, and then the auditing of how data is used itself can be incredibly granular and incredibly detailed. and i want to get to other point. this is a lot here. just these two capabilities are a significant improvement of what existed before and can get us a long way. and there are things that exist today. now i'm obligated to say that of course, this is done the best, but not exclusive to pollentier, and they can be deployed and can be

the second stage of the pipeline merges the data. if two data files can be determined to correspond to the same person, for example, because they both contain the same unique identifier, then those files can be merged. and merging can create an avalanche effect, because merged files convey more precise information act identity and behavior and that precision in turn allows further merging. one file might contained detailed information act behavior, and another might contain precise identity information. merging those files links behavior and identity together. the third stage of the pipeline uses big data methods such as predictive analytics to infer facts about people. one famous example is when the retailer target used purchases of the products such as skin lotion to infer pregnancy. today's machine learning methods often enable sensitive information to be inferred from seemingly less sensitive data. inferences also can have an avalanche effect because each inference becomes another data point to be yutzed in making further inferen

the data with us. the database includes both removals and a certain number of informal returns. our report and my presentation today focus exclusively on formal removals. we're taking the returns out of the conversation. we also used dhs data to model the nonremovals that don't appear in the data set that are in the dhs data set because they were also removed. that's basically cpv removals to mexico and canada, nonjudicial removals to mexico and canada. and then throughout the report and my presentation the data measure removal events, not unique individuals so, what that means is some people are deported more than wasn't and they appear more than one time in the data set so it's a certain bias in the data toward these sort of repeat customers. all together, 3.7 million people have been removed between 2003 and 2013, so that's the universe that we're looking at over that 11-year period. just sort of a basic profile of who's been deported during that period, fornally removed during that period, 91% of r

on the data that it collects. any trends in pay discrimination, the number of investigationses it commence based on the reports that it receives and the number of contractors that were penalized for violating equal pay laws. discrimination based on sex and race is already prohibited in the city's administrative code and by federal law. however, the human rights commission only enforces the provisions at this point when individuals file allegations of unequal pay. the problem is that it is simply impossible for women to determine whether they are victims of wage discrimination because, as a 2011 survey from the institute for women's policy research found, about half of workers report that the discussion of wage and salary information is either discouraged or actually prohibited in the workplace. there is essentially a culture of secrecy around compensation which has allowed this wage disparity to remain countrywide for more than 50 years. this legislation is groundbreaking because it provides the human rights commi

well being assessment data and the benchmark data. the characteristics and i cannot stress this enough is that all data will be identified. we will not be seeing any employee level data of identifying any employee at all. it will be confidential. and the well being assessment data annually. as far as the robust analytics we'll be able to do our analysis on the wellness of the diseases and programs and certainly not only cost and utilization importance to us but the quality of care our members receive is very important to us and we'll be able to use database guidelines for those gaps of care and we'll be using our ad hoc analyses for audits and clinical procedures for ncqa and we'll be able to integrate our plans to provide those integrated dash boards. at this point i would like to turn the presentation to my colleague david flores. >> i get the short part of this presentation and i'm very very pleased about that. president fraser, commissioners, we've given you the background for our clearance data base and now we are at the nuts and

to manage the data. how do you support and not how do you replace the policy. totally accepting your foint about focussing on risk and use, you are not saying that collection is irrelevant and the fourth amendment is a collection limitation. that company that has a flashlight is collecting data from -- nobody even got to the analysis and that was inappropriate in and of itself. >> you are right and i agree completely. i am not suggesting collection is irrelevant, but it's the end of the story so once you across a spill way in a dam, once you are over the limit, anything else goes. you have your collection authorization and your retention and use and dem nation and each one of those. i'm not doubting what chris is saying, but when i read the report that came out from the nsa office, a well-written report was full of all the limits on what they were doing. the incredible bureaucracy on that. it sort of ignored the factors that struck most to the american people. how is the authorization obtained in the fir

the fatality system, the data there is only based on data from reports completed in the field by police officers and they in most cases i don't believe they make a determination between a distinction between whether a driver was known to be taintive or whether they simply couldn't tell whether the driver was drousey or not. there is actly an indication in whether the driers' condition at the time of the crash was unknown. however the proportion in which they indicate the driver's condition was unknown is imapplauseably low even among only one which driver was involved, or where the driver was deceased at the time. they only report that the driver's condition is unknown. in more than 60% of the cases they report no contributing factors apparent and i'm not sure how one could arrive that determination when the only person involved is already deceased at the time they arrived. >> i want to comment on the notion of a prevalence changing. there is no question that the introduction of rumble strips and airbags may have made it safer or less likely to be lethal for somebody who drives drowsy.

the use of the aggregated data. you can't do it at collection databases are there. they are so vague it is impossible to stop unless you're going to keep google from collecting. we are going to have a data collection. >> just a quick note, i remember -- agree with most of what has been said. is tor way to look at it say the information that is being gathered and the using normal powers of observation would be in someone's control. that the point of collection is a moot point. the mere fact that google has all of this information and facebook has all of this information does not mean the government has all this information. there are new tech knowledge he's -- technologies. there are still plenty of room to regulate at the collection point. i will be very brief as well. the other what panelists have said, i would point out that much of the information of that is in corporate databases is information that was observed rather than disclosed. consent.not always it is very thin from the person who the data is about. i do

data and we could put this on the internet. you know? and sort of under our name and in our system as something that can be, you know, built on that is not necessarily endorsed by the government, but it is part of our organization's effort to make the data more understandable and usable and so we are working on some very, you know, sort of basic tools for doing that. but, it is early stages, but it is a focus of ours. >> commissioner king? >> you have done a good job in allowing the general public to get into the weeds of things like a general election and in regards to funding an election and who is involved, and the whole aspect of following the money which is or tells so much about government, and the transparency and the lack of transparency of government and that is enormously valuable and i want to ask you a question which may be unfair and if it is, feel free not to answer it. since you have gotten into the weeds and you have seen, how elections and balloting works and where the money is coming from and where it is comin

here's the kind of data we gary gather here's other data about you. that's no so the bad if you're honest. you can have abbreviated knew nutrition facts and maybe there's a longer version below. is a mentioned, credit cards and some financial statements you -- some of mutual fund statements have simplified what they do with fees. i don't -- i think that honesty should be rewarded by customers itch think there's a lot of customers who are frustrated by the deliberate obfuscation of what the companies are doing, and they're crying out for something simpler and your study you just mentioned is an example of that. please, over here. >> block planning to write something understandable, i think it's the way to make you just skip to the end and say, i agree, and then whatever you need to do, whether it's to buy something on amazon or whatever, then you're just agreeing to -- first of all you know that you will not understand it, unless you are a ph.d, and then i guess the -- exactly not to make it understandable, so that people just skip and go to the end. o

and that the hope sf, the hunter's view and the data would be very important to analyze from that. because it will help us to calibrate our student yields and the future mixed inside housing. and the bottom line is, that the enrollment forecast, and they did anticipate and they suggested that while the numbers may vary and i will move to the next slide, the trend they imagined being the same and in other words, that the cohorts that housing and increased birth rates will result in over a period of time, an increase in enrollment for sfusd and again, just to kind of reiterate what was kind of forshadowed in the earlier discussion and presentation, that the forecasts are highly sensitive to the student yields and that is something that we need to monitor and the timing and when they develop these enrollment projections the timing about housing development was uncertain and so moving forward and working with the planning department and others, it will be very helpful to have the most current data to refresh the enrollment forecast and this visual on the next slide just kind of shows,

they get data from the san francisco planning department and the housing authority and the office of community and ininvestment and these are the what they worked with and got the data from and at the treasure island, the u.s. census data and what they do and the method that they do is look at birth friends, in san francisco, and the housing growth and a key element of that, is work around, looking at the student yields and so i thought that this afternoon i will go a little bit deep on the methods that they used for the students yield because i think that is key to the discussion and how we are using that information for the forecast. and so, the number of births has increased, and we are talking about births and we are talking about residents of san francisco and it increased during the starting this decade, and last decade, resulting in an increasing enrollments in k12 that started increasing from the low point in 2008 and most of the enrollment changes from since 2008 canen explained by changes in birth numbers, the birth numbers have been an excellent predicter of subsequent enr

the telephone companies keep the data. but the point is there is a 90-day period that requires the telephone companies to turn over the information from their records to the government for purposes of this program. now, the government may then retain it for five years, but that's not the same as asking whether the telephone company must keep it for five years. this is an order from a court requiring turnover for a 90-day period. that's the relevant consideration for purposes of -- >> why is that relevant consideration rather than -- how can we discard the five-year term that government keeps it? >> because under smith and miller, the question is, who's information is this and is there an expectation of privacy by another person? and here the information belongs to the telephone company just as the financial information in miller belonged to the bank. and the fact is once the bank has them or the telephone company has them, it can turn them over to the government. that's the riskier taking. >> phone company ownership has an

i basically just went on to the data s.f. and put in specific terms looking 30 years ago or older and then also certain class codes [speaker not understood] felt would be best that fall under the definitions of it and then just compile it had into an excel spreadsheet. you only have one class code. i did a couple more and i hope to continue to work with that project because i really think it's a great project and would like to provide any help that i can. >> thank you. >> so, thank you, stephanie, and monique. so, you're welcome to stay or you're welcome to go back downstairs if you like. all right. also, so, to move on, so, the a-d-a subsidized assessment program we are now going to expand it city-wide. and it will be available as long as there are funds annually because of the dollar registration that's added to the annual business registration. we have approximately $50,000 to work with. so, that could add 40, around 40 kass p, subsidized casp inspections. it was opened up to targeted certain neighborhood programs. now wit

important but a advantage with the hand signals for the pedestrian walking so what did the data stlils at those locations the data shows the intersection blocking there's no room once a vehicle a there and issuing citations to use hand signals are most effect about a 10 percent advantage as i mentioned for intersection if they're using hand signals i think about that it makes sense to the people on the ground helping people passing this is informational not a data point but information in costs that generates the activity this is this is a little blue signal but the cost of running the pilot with our interim staff and our parking control officers was a total of about $270,000 for about $4,500 a shift fairly expensive the income we got from the citations we issued was $9,000 it was $10,000 for each citation the state gets $9 of those funds and we keep the rest for mta operating budget it is not able we didn't issue citation everyday we didn't issue as many sizing as the cost of the pilot would have been so in summary what we saw was that enforcement works the data shows intersection and

. >> the data from 2009 was the last data cited in official publications by the national traffic safety dministration. that data -- those statistics are based on data that is collected annually and is presently available through 012. the most recent data from 2012 shows that 1.6% of drivers involved in fatal crashes in 2012 were believed to be drowsy or fatigued at the time of the crash. >> what would you say is the general trend for the drowsy driving data? is there a trend and, if so, which direction is it heading? >> in official statistics, there does not seem to be much of a general trend. if anything, the proportions in the official statistics may have decreased slightly. as i talk about in my presentation, i believe that is a vast underestimate. i would not look to the data published in those sources for evidence of a trend or lack thereof. in the study that i conducted that looks at eta from 1990 99 - data from 1999-2008, in that data, there was not evidence of an increasing or decreasing trend in the proportion of the crashes that involved drowsiness. i would note that, in recen

that data after the fact. second, this was discussed in the first panel, but there's a misconception, i think, that the phipps are primarily useful for commercial privacy. in my written testimony i talk about the fact that the phipps remain a critical benchmark against which to measure the privacy impacts of counterterrorism policies and i'll just add, given the previous discussion that literally since their inception in 1973, the committee that road the report dedicated a section talking about how, of course, not all of the phipps can apply in the intelligence context but clearly some of them must because the risk is too high. third, and my testimony i talk about that we need to remember that privacy is not about taking but -- pardon me, that it's about taking and not about sharing and fourth and finally, i think that americans do expect a degree of privacy in public. now, given my limited time here, i want to focus my oral testimony on just that first point, collection. i think it's the most important. after

data, electoral data, and crunch that on the backend. once we do that, we are able to look at things like, what is the profile of this individual legislator? how has he or she voted in the last 20 years? or happens to be a bill that relates to something they have looked at before. can we go back into the data and extrapolate into the future, and come with those outcomes? like this is done via an algorithm you developed? how old are you? >> i am 22. >> how did you get interested in all of this? likes my own career has sort of taken a lot of twists and turns in the last couple of years. in they started off obama campaign in 2008, working in the field and then moving to the data side, initially looking at some of the more demographic type issues, those types of things, but moving more into issue-based things. after the campaign ended, i was thinking of what i could do. i was 17 at the time. and ended up really deciding to throw my hat into the race for a board of education seat that came up in maryland. and to my surprise, i won, the stone

users of the data could be charged to use the data and they could pay for that data with dollars or in contributions of their own data into the system. in terms of it is already being done, those are microcosmic by comparison, not at systematic testing, i remember 23, an mri, no one is testing those and so forth and fluids and as i said, this is a mother ship in which other concepts can dock and trade data and that is why after being funded, managed in some regard by the federal government. >> i was originally going to comment on dr. hanlon's comments but there's a lot of rich discussion. one additional would make that has already been said more broadly is whether it is a big longitudinal study or a smaller study taking advantage of existing data what is critical is we know which questions need to be asked and we designed the study in such a way that would get meaningful answers to tell us what we need to know and that is so important and there's a lot going on in the area of clinical trial designs, trying to ask and answer questions in ways that require smaller numbers and shorter tri

>> the data continues to be provided to us and now to access the data we have to go to the court to get permission to access that data. >> but why continue to gather the bulk metadata if the administration and he and i don't think this is the best approach? >> i guess i'm confused because i don't think i've heard heard a present or the dni say that the axis to the data is not of value. what i think i have heard is the question gets to be who should hold the data but the president directed in his remarks on the 17th of january we will continue to implement the programs while the congress works to how we will make long-term changes. we'll continue to do that on the 90-day intervals of every 90 days now is to go back and asked for continued permission. >> one last comment, if the administration believes and i understand that they do, but the better model is to go to a paradigm where the companies hold onto their own data it doesn't make sense for us to continue the collection of bulk metadata. we are not legally required to and there's no reason not to move to that model and make that tran

and that is like - brings all the data into question. as a scientist we want to make value judgment. they don't give you accuracy. >> could it be a difference of strides. >> i'm bringing the same amount of calories. it doesn't spend a lot of time educating you on what the data means and how to interpret it. >> i tried out the fit bit. it was higher in terms of the step count. you can check it. you can watch it tally up the steps. you are trying to develop a lifestyle of habits. using this and the app. from green stadiums to birds with goggles, i check my pulse. talking about the science, my heart is beating a little faster. for more exciting innovations join us next time on "techknow". go behind the scenes at aljazeera.com. >> like most people, i'm not an activist by nature. there's really not that many people whose greatest desire is to go out and fight the system. my theory of change was i'll

somebody referred to the modern era, the era of the internet of things and big data, ubiquitous data flows as the golden age of surveillance. and it seems to me that both trends will always be there. more and more information available to the private sector and possibly to the government, and increasing pervasiveness, or at least increasing diffusion, diffusion of encryption. comments on that as a premise, first of all, the premise of my question. am i right? and secondly where does that leave the government? do you agree with my assumption that there will still be huge amounts of information available to the private sector for its purposes as well as to the government but let's go right down the road. professor. >> so, i believe that there will still be a lot of data that's available for government. when i say that i really support encryption by default, i also really think that our country really, we were the code hackers. it was really critical in world war ii. and i think that instead of, or taking a lazy approach, and taking the that the backdoor, we should just get better as sm

take espilion, a giant in the business, the holder of credit and marketing data. in a maim scror lapse, that brian crabb reported, an identity thief in vietnam gained access to a dater base containing personal information from 200 million americans from a company opened by xpirion. >> it was selling information, they claim unknowingly, i'm willing to give them the benefit of the doubt to an individual who said he was a u.s. based private investigator. they refused op on-air interview but said: >>> to brian crabbs, the indoor question razes questions. >> it's a recipe for disaster when an organization with almost no accountability collects sensitive and voluminous information on people. when they have a security incident that jeopardizes the security of that information, consequences. >> a company is attempting to answer critics concerns. in an industry first data broker axiom launched about the data.com which let's you see what it knows about you. we found out what they know about me. >> i see your date of birth, male, african american, completed graduate school.

this is the first of the november data. we finished october so you will be seeing me a lot in the next 15 minutes. back to you. >> rick, thank you very much. rick santelli. tomorrow the 88th macy's thanksgiving day parade televised nationwide on nbc at 9:00 a.m. fresh from the opening bell joined by amy the executive producer of the parade and joins us at post nine. great to have you back. >> great to be here. >> spoke to you last year as well. >> yes. >> talking about weather of course. >> yes. >> big whatever storm cato is that a waworry tomorrow some. >> no everybody we have spoken to told us the weather will clear for tomorrow. inflating our balloons tonight. do that all day. tomorrow they will fly high. >> wow us with numbers, numbers of floats. >> this is a parade of numbers. 27 floats, we've got 16 giant balloons, we've got 12 marching bands one drum corps, 1,000 clowns, 1,000 balloon handlers and 1200 leaders and dancers and 3 million people lining the parade route and 50 million watching on television. >> i think t