- LegiNote Project Development Story 2 - Tech Stack and Workers
- The second part of the LegiNote project development story, focusing on worker development using the Go language. It covers the implementation of data collection and update logic, along with the project's structure.
This post has been translated by AI.
Post summarized by durumis AI
- During the development of the LegiNote project, we implemented a process to retrieve bill information using the National Assembly OpenAPI.
- We used the Go language to request data through HTTP Request and parsed the received JSON data into structs.
- This article explains the process from API key issuance to data request and parsing, and explores potential future improvements.
Hello, this is StatPan.
I'm writing about the development process of the LegiNote side project.
Please refer to the following link for the previous episode.
Worker 1: worker-bill
The name 'bill' sounds familiar, like a foreign friend's name. I got the inspiration from the meaning of the word below.
Previously, I searched for various English words related to the National Assembly domain to find appropriate variable names and repository names. I applied what I found then.
Meaning of the word 'bill'
The OpenAPI specification of the target location where the Worker will collect data is as follows.
Input Parameters
Bill Review and Processing API Input Parameters
Output Parameters
Bill Review and Processing API Input Parameters
It seems to provide a lot of data and can offer quite detailed information.
HTTP Request Essentials
First, let's look at what is needed to make the HTTP request with the input parameters above.
URL, HTTP request method(although not shown here, it's GET, POST is not allowed), KEY, Type, pIndex, pSize values are required.
For the KEY value, similar to other OpenAPIs, you need to obtain a separate authentication key.
Since this is a development story, I wanted to provide a link to a reference... but unfortunately, it's still relatively minor, and there are no articles on Google explaining how to obtain the authentication key. So, for now, let me briefly explain it with images...
Home Screen
Click the My Page button on the home screen.
My Page
API Key Issuance Screen
In the above screen, for the purpose or content, I simply wrote that it was for the purpose of a public data competition. You don't need to worry too much about filling out the content. As long as it's not a few lines of inappropriate reasons, I think they'll approve it right away.
The key you obtain will then issue the KEY needed for the HTTP explanation above.
Let's send a request to the URL above (I hope they will include the method in the next update).
go```go package main import ( "encoding/json" "fmt" "io" "log" "net/http" "net/url" ) func procErr(e error, msg string) { if e != nil { log.Fatal(e, msg) } } func main() { rawUrl := "https://open.assembly.go.kr/portal/openapi/TVBPMBILL11" parsedUrl, err := url.Parse(rawUrl) procErr(err, "err after parsedUrl") params := url.Values{} KEY := "your_key" //change this procErr(err, "err after req, 27") req.Header.Add("Content-Type", "application/json") req.Header.Add("User-Agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36") params.Add("KEY", KEY) params.Add("Type", "json") // JSON 좋.아 params.Add("pIndex", "1") // 이건 왜 integer로 표시했는지 모르겠지만 string params.Add("pSize", "1") client := &http.Client{} reqURL := fmt.Sprintf("%s?%s", rawUrl, params.Encode()) req, err := http.NewRequest(http.MethodGet, reqURL, nil) res, err := client.Do(req) procErr(err, "err, after client do") defer res.Body.Close() var jsonMap any response, err := io.ReadAll(res.Body) procErr(err, "err after read response") err = json.Unmarshal(response, &jsonMap) procErr(err, "err after json unmarshal") fmt.Println(jsonMap)
```
I was able to confirm that it works using the code above.
A tip for you: you must add the Header if you want to use this API smoothly. (Semi-threat)
After that, fill in the essential parameters for the GET URL.
The values for pIndex and pSize are supposed to be integers, but they work even if you apply the string type. (I don't understand why).
You can also use the XML format for Type, but I chose JSON.
Result
map[TVBPMBILL11:[map[head:[map[list_total_count:103305] map[RESULT:map[CODE:INFO-000 MESSAGE:정상 처리되었습니다.]]]] map[row:[map[AGE:22 BILL_ID:PRC_T2U4C0A8B2Z0A1Y3Y5G2H0F0G5E4F4 BILL_NAME:정치자금법 일부개정법률안 BILL_NO:2203179 CMT_PRESENT_DT: CMT_PROC_DT: CMT_PROC_RESULT_CD: COMMITTEE_DT: COMMITTEE_PROC_DT: CURR_COMMITTEE: CURR_COMMITTEE_ID: LAW_PRESENT_DT: LAW_PROC_DT: LAW_PROC_RESULT_CD: LAW_SUBMIT_DT: LINK_URL:https://likms.assembly.go.kr/bill/billDetail.do?billId=PRC_T2U4C0A8B2Z0A1Y3Y5G2H0F0G5E4F4 PROC_DT: PROC_RESULT_CD: PROPOSER:고동진의원 등 10인 PROPOSER_KIND:의원 PROPOSE_DT:2024-08-23 RST_MONA_CD:HS39431V RST_PROPOSER:고동진] map[AGE:22 BILL_ID:PRC_O2M4M0L8L2T0R1S3Q5R1P4Q0Y2W2X6 BILL_NAME:정당법 일부개정법률안 BILL_NO:2203178 CMT_PRESENT_DT: CMT_PROC_DT: CMT_PROC_RESULT_CD: COMMITTEE_DT: COMMITTEE_PROC_DT: CURR_COMMITTEE: CURR_COMMITTEE_ID: LAW_PRESENT_DT: LAW_PROC_DT: LAW_PROC_RESULT_CD: LAW_SUBMIT_DT: LINK_URL:https://likms.assembly.go.kr/bill/billDetail.do?...
You can get the result as shown above.
Separately, to directly control that object, I defined all the structs and retrieved the values, which also involved numerous debugging sessions.
Due to character limitations, I think I'll have to continue with unmarshalling this value into a struct in the next episode.
One request took up a whole episode during the development process. Of course, due to the nature of using this OpenAPI, it took a lot of time to utilize the API and establish the struct structure. I need to find a way to prevent others from experiencing the same pain.
If you encounter any issues with the above request not working as expected or have any complaints about my strange code, please bring me some cigarettes.