mirror of
https://github.com/hpware/news-analyze.git
synced 2025-06-24 05:24:23 +00:00
Compare commits
No commits in common. "6f759cb61292c1b86c951131d286541abf901d13" and "731d4d9a8ce42b2b2dc8bc97f3c61e745715b881" have entirely different histories.
6f759cb612
...
731d4d9a8c
23 changed files with 241 additions and 88 deletions
29
.dev.env
Normal file
29
.dev.env
Normal file
|
@ -0,0 +1,29 @@
|
|||
# For prod use please use the .env.example file.
|
||||
# Please use .dev.env as an starting point. Rename it to .env and fill in the values, the application needs it.
|
||||
|
||||
# This is the developmemnt use .env file.
|
||||
|
||||
# S3 INFO
|
||||
S3_ACCESS_KEY=""
|
||||
S3_SECRET_KEY=""
|
||||
S3_BUCKETNAME=""
|
||||
S3_ENDPOINT=""
|
||||
|
||||
# GITHUB OAUTH (NOT WORKING 4n)
|
||||
NUXT_GITHUB_CLIENT_ID=""
|
||||
NUXT_GITHUB_CLIENT_SECRET=""
|
||||
|
||||
# GLOBAL DATABASE
|
||||
POSTGRES_URL=""
|
||||
|
||||
# GROQ API KEY
|
||||
GROQ_API_KEY=""
|
||||
|
||||
# PASSWORD SALT
|
||||
PASSWORD_HASH_SALT=""
|
||||
|
||||
# CF TURNSTILE
|
||||
NUXT_CF_TURNSTILE_SITE_KEY=""
|
||||
NUXT_CF_TURNSTILE_SECRET_KEY=""
|
||||
|
||||
NUXT_DEV_ENV=true
|
|
@ -1,4 +1,5 @@
|
|||
# Please use .env.exmaple as an starting point. Rename it to .env and fill in the values, the application requrires it.
|
||||
# For development use, please use the .dev.env file.
|
||||
# Please use .env.exmaple as an starting point. Rename it to .env and fill in the values, the application needs it.
|
||||
|
||||
# This is the default .env file.
|
||||
|
||||
|
|
BIN
.github/OTHER/ig_story_58m.png
vendored
BIN
.github/OTHER/ig_story_58m.png
vendored
Binary file not shown.
Before Width: | Height: | Size: 1.4 MiB After Width: | Height: | Size: 2 MiB |
|
@ -19,7 +19,7 @@ https://yhw.tw/news
|
|||
|
||||
## Issues:
|
||||
### Onboarding:
|
||||
Onboarding is a must for most people that are using the app for the first time, but I want to do to via a non-video like system, however implementing the function in a already large repo is kinda hard. So later this week, I will just add a basic video onboarding system.
|
||||
Onboarding is a must for most people that are using the app for the first time, but I want to do to via a non-video like system, however implemnting the function in a already large repo is kinda hard. So later this week, I will just add a basic video onboading system.
|
||||
|
||||
### User actions via the API:
|
||||
Currently, user actions are broken.
|
||||
|
@ -36,7 +36,7 @@ Chatbot, which is chatbot for chatting about news articles, is currently not ava
|
|||
### Server Downtime
|
||||
Use https://status.yhw.tw/ for checking down time, most of the time it will be up, but sometimes it just won't updated to the latest feature & update.
|
||||
|
||||
#### Archive:
|
||||
Archive:
|
||||
I fixed most issues of the server, including the nameserver stuff, if you want to know how I fixed it, you can view how I fixed it [here](/server_fixes.md) or on [My broken blog](https://4-1-2.yuanhau.com/posts/)
|
||||
|
||||
### Scraping restrictions:
|
||||
|
@ -51,12 +51,9 @@ A few pages now contains translations, like the news, aboutNewsOrg and newsView
|
|||
### Deploying:
|
||||
This code is absolutly NOT designed to be spinned up at Vercel or Netlify, it has the scraping system now inside of the main website code, oh also the entire "caching feature" is based in memory, so please don't use those platforms, for Zeabur your cost might be expensive. idk, I haven't tried hit yet. The web url: https://news.yuanhau.com is hosted on my own infra, you should too. Please get a server off of yahoo 拍賣, 蝦皮 or eBay to do so.
|
||||
|
||||
### The API returning outdated data from more than 5+ years:
|
||||
Here is the GitHub Issue: https://github.com/hpware/news-analyze/issues/2
|
||||
|
||||
## Why?
|
||||
|
||||
We'll use this news article from May 7th 2025 as an example:
|
||||
We'll use this news article as an example:
|
||||
|
||||
```
|
||||
Zhu Lilun criticizes the government for being like Hitler German Institute in Taiwan: History should not be distorted for politics | Politics - CNA
|
||||
|
|
70
components/app/windows/hotnews.vue
Normal file
70
components/app/windows/hotnews.vue
Normal file
|
@ -0,0 +1,70 @@
|
|||
<script setup lang="ts">
|
||||
// Great, there are now no errors ig
|
||||
const emit = defineEmits(["windowopener", "error", "loadValue"]);
|
||||
const props = defineProps<{
|
||||
values?: string;
|
||||
}>();
|
||||
import DraggableWindow from "~/components/DraggableWindow.vue";
|
||||
const ffeed = ref();
|
||||
import Button from "~/components/ui/button/Button.vue";
|
||||
const pending = ref();
|
||||
|
||||
try {
|
||||
const { data, pending } = await useFetch("/api/cached/rss/google");
|
||||
ffeed.value = data.value;
|
||||
} catch (error) {
|
||||
console.error("Error:", error);
|
||||
}
|
||||
</script>
|
||||
<template>
|
||||
<div v-if="!ffeed">Loading...</div>
|
||||
<div
|
||||
v-for="item in ffeed"
|
||||
class="justify-center align-center text-center p-4 border border-black rounded-lg m-4"
|
||||
>
|
||||
<span class="text-xl text-bold text-gray-900"
|
||||
>{{ item.title }}
|
||||
<!--<span
|
||||
v-if="ass.some((app) => item.title.includes(app))"
|
||||
class="text-red-500 text-sm"
|
||||
>
|
||||
- 疑似來自有中資背景公司
|
||||
</span>-->
|
||||
</span>
|
||||
<h4 class="text-gray-500 text-sm">
|
||||
{{ new Date(item.date).toLocaleString() }}
|
||||
</h4>
|
||||
<div class="flex justify-center gap-2 mt-1">
|
||||
<NuxtLink :to="item.link" target="_blank">
|
||||
<Button>文章</Button>
|
||||
</NuxtLink>
|
||||
<NuxtLink>
|
||||
<Button>關於媒體</Button>
|
||||
</NuxtLink>
|
||||
</div>
|
||||
<br />
|
||||
類似新聞:
|
||||
<div v-for="itit in item.content">
|
||||
<ul v-for="ititit in itit">
|
||||
<li v-if="ititit.content?.[0].content[0] !== item.title">
|
||||
-
|
||||
<a :href="ititit.content?.[0].attributes?.href" target="_blank">{{
|
||||
ititit.content?.[0].content[0]
|
||||
}}</a>
|
||||
-
|
||||
<a :href="'/find/newsOrg?name=' + ititit.content?.[2].content[0]">{{
|
||||
ititit.content?.[2].content[0]
|
||||
}}</a>
|
||||
<!--<span
|
||||
v-if="
|
||||
ass.some((app) => ititit.content?.[2].content[0].includes(app))
|
||||
"
|
||||
class="text-red-500 text-sm"
|
||||
>
|
||||
- 疑似來自有中資背景公司
|
||||
</span>-->
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</template>
|
|
@ -62,24 +62,6 @@ const submitCustomApiKey = async () => {
|
|||
return;
|
||||
}
|
||||
}
|
||||
try {
|
||||
const req = await fetch("/api/user/submitGroqKey", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify({
|
||||
value: customApiKey.value,
|
||||
}),
|
||||
});
|
||||
|
||||
const response = await req.json();
|
||||
if (response.error) {
|
||||
console.error("Error updating user data:", response.error);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("Failed to submit change:", error);
|
||||
}
|
||||
};
|
||||
|
||||
const checkValidApiKey = () => {
|
||||
|
@ -98,6 +80,11 @@ const confirmDelete = async () => {
|
|||
showDeleteDialog.value = false;
|
||||
};
|
||||
|
||||
const deleteAccount = async () => {
|
||||
const req = await fetch("/api/user/action", {
|
||||
method: "DELETE",
|
||||
});
|
||||
};
|
||||
const apiKey = customApiKey.value;
|
||||
try {
|
||||
const sendApi = await fetch("/api/ai/loadCustomGroqApi", {
|
||||
|
|
|
@ -1,4 +1,5 @@
|
|||
<script setup lang="ts">
|
||||
import logoutUser from "~/components/logoutuser";
|
||||
// Imports
|
||||
const { t, locale } = useI18n();
|
||||
// Values
|
||||
|
|
14
components/loadUserInfo.ts
Normal file
14
components/loadUserInfo.ts
Normal file
|
@ -0,0 +1,14 @@
|
|||
export default async function loadUserInfo() {
|
||||
return {
|
||||
langPref: "en",
|
||||
doNotShowLangPrefPopUp: false,
|
||||
email: "test@yuanhau.com",
|
||||
name: "Howard",
|
||||
useCustomGroqKey: true,
|
||||
translate: {
|
||||
enabled: true,
|
||||
lang: "en",
|
||||
provider: "google",
|
||||
},
|
||||
};
|
||||
}
|
3
components/logoutuser.ts
Normal file
3
components/logoutuser.ts
Normal file
|
@ -0,0 +1,3 @@
|
|||
export default function logoutuser() {
|
||||
return;
|
||||
}
|
20
components/newsAnalyzer.ts
Normal file
20
components/newsAnalyzer.ts
Normal file
|
@ -0,0 +1,20 @@
|
|||
// News Analyzer Class
|
||||
class NewsAnalyzer {
|
||||
private sensitivePatterns: RegExp[];
|
||||
constructor() {
|
||||
this.sensitivePatterns = [];
|
||||
}
|
||||
|
||||
isKidFriendly(title) {
|
||||
for (let pattern of this.sensitivePatterns) {
|
||||
if (pattern.test(title)) return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
public setSensitivePatterns(patterns: RegExp[]): void {
|
||||
this.sensitivePatterns = patterns;
|
||||
}
|
||||
}
|
||||
|
||||
export default NewsAnalyzer;
|
1
database/README.md
Normal file
1
database/README.md
Normal file
|
@ -0,0 +1 @@
|
|||
# 資料庫資訊 Database info
|
|
@ -21,8 +21,6 @@
|
|||
"色誘",
|
||||
"死亡",
|
||||
"撩妹",
|
||||
"裸上身",
|
||||
"曬辣",
|
||||
"辣媽"
|
||||
"裸上身"
|
||||
]
|
||||
}
|
4
pages/botcheck.vue
Normal file
4
pages/botcheck.vue
Normal file
|
@ -0,0 +1,4 @@
|
|||
<template>
|
||||
<!--TODO: Cover the filtering out with a actucal bot check, think anubus but it helps load the website algrothom? and helps client load faster? and add help via an api that LLMs can use?-->
|
||||
<div></div>
|
||||
</template>
|
|
@ -47,8 +47,12 @@ import { gsap } from "gsap";
|
|||
import confetti from "js-confetti";
|
||||
import translate from "translate";
|
||||
|
||||
// Import Components
|
||||
import loadUserInfo from "~/components/loadUserInfo";
|
||||
|
||||
// Import Windows
|
||||
import UserWindow from "~/components/app/windows/user.vue";
|
||||
import HotNewsWindow from "~/components/app/windows/hotnews.vue";
|
||||
import SourcesWindow from "~/components/app/windows/sources.vue";
|
||||
import AboutWindow from "~/components/app/windows/about.vue";
|
||||
import ChatbotWindow from "~/components/app/windows/chatbot.vue";
|
||||
|
@ -101,6 +105,7 @@ const translateProvider = ref("");
|
|||
|
||||
// Key Data
|
||||
const menuItems = [
|
||||
// { name: t("app.hotnews"), windowName: "hotnews" },
|
||||
{ name: t("app.news"), windowName: "news" },
|
||||
{ name: t("app.sources"), windowName: "sources" },
|
||||
{ name: t("app.starred"), windowName: "starred" },
|
||||
|
@ -108,10 +113,20 @@ const menuItems = [
|
|||
{ name: t("app.about"), windowName: "about" },
|
||||
{ name: t("app.terminal"), windowName: "tty" },
|
||||
{ name: t("app.settings"), windowName: "settings" },
|
||||
{ name: t("app.login"), windowName: "login" },
|
||||
{ name: t("app.leave"), windowName: "leave" },
|
||||
];
|
||||
|
||||
const associAppWindow = [
|
||||
{
|
||||
name: "googlenews",
|
||||
id: "1",
|
||||
title: t("app.hotnews"),
|
||||
component: HotNewsWindow,
|
||||
width: "700px",
|
||||
height: "500px",
|
||||
translatable: true,
|
||||
},
|
||||
{
|
||||
name: "login",
|
||||
id: "2",
|
||||
|
@ -591,7 +606,6 @@ const toggleTranslate = (windowId: string) => {
|
|||
const translateAvailable = () => {};
|
||||
|
||||
// Load user config via HTTP requests to the server.
|
||||
/*
|
||||
onMounted(async () => {
|
||||
const loadUserInfoData = await loadUserInfo();
|
||||
if (!loadUserInfoData.user) {
|
||||
|
@ -609,7 +623,7 @@ onMounted(async () => {
|
|||
// Use Google as the default translate provider
|
||||
translateProvider.value = loadUserInfoData.translate.provider || "google";
|
||||
console.log(langPrefDifferent);
|
||||
});*/
|
||||
});
|
||||
</script>
|
||||
<template>
|
||||
<div v-if="changeLangAnimation">
|
||||
|
|
2
public/agents.txt
Normal file
2
public/agents.txt
Normal file
|
@ -0,0 +1,2 @@
|
|||
# What does this website do?
|
||||
This website mainly do news compare & news analyze stuff.
|
BIN
public/fun-bg.jpg
Normal file
BIN
public/fun-bg.jpg
Normal file
Binary file not shown.
After Width: | Height: | Size: 1.8 MiB |
1
public/logos/GitHub-dark.svg
Normal file
1
public/logos/GitHub-dark.svg
Normal file
|
@ -0,0 +1 @@
|
|||
<svg xmlns="http://www.w3.org/2000/svg" width="1024" height="1024" fill="none" viewBox="0 0 1024 1024"><path fill="#ffff" fill-rule="evenodd" d="M8 0C3.58 0 0 3.58 0 8C0 11.54 2.29 14.53 5.47 15.59C5.87 15.66 6.02 15.42 6.02 15.21C6.02 15.02 6.01 14.39 6.01 13.72C4 14.09 3.48 13.23 3.32 12.78C3.23 12.55 2.84 11.84 2.5 11.65C2.22 11.5 1.82 11.13 2.49 11.12C3.12 11.11 3.57 11.7 3.72 11.94C4.44 13.15 5.59 12.81 6.05 12.6C6.12 12.08 6.33 11.73 6.56 11.53C4.78 11.33 2.92 10.64 2.92 7.58C2.92 6.71 3.23 5.99 3.74 5.43C3.66 5.23 3.38 4.41 3.82 3.31C3.82 3.31 4.49 3.1 6.02 4.13C6.66 3.95 7.34 3.86 8.02 3.86C8.7 3.86 9.38 3.95 10.02 4.13C11.55 3.09 12.22 3.31 12.22 3.31C12.66 4.41 12.38 5.23 12.3 5.43C12.81 5.99 13.12 6.7 13.12 7.58C13.12 10.65 11.25 11.33 9.47 11.53C9.76 11.78 10.01 12.26 10.01 13.01C10.01 14.08 10 14.94 10 15.21C10 15.42 10.15 15.67 10.55 15.59C13.71 14.53 16 11.53 16 8C16 3.58 12.42 0 8 0Z" clip-rule="evenodd" transform="scale(64)"/></svg>
|
After Width: | Height: | Size: 963 B |
30
server/api/cached/rss/google.ts
Normal file
30
server/api/cached/rss/google.ts
Normal file
|
@ -0,0 +1,30 @@
|
|||
import Parser from "rss-parser";
|
||||
import { HTMLToJSON } from "html-to-json-parser";
|
||||
|
||||
export default defineEventHandler(async (event) => {
|
||||
let array = [];
|
||||
const parser = new Parser();
|
||||
try {
|
||||
const feed = await parser.parseURL(
|
||||
"https://news.google.com/rss?&hl=zh-TW&gl=TW&ceid=TW:zh-Hant",
|
||||
);
|
||||
feed.items.forEach(async (item) => {
|
||||
const rawRelatedNews = await HTMLToJSON(item.content, true);
|
||||
const relatedNews = JSON.parse(rawRelatedNews.replace("ol", ""));
|
||||
array.push({
|
||||
title: item.title,
|
||||
link: item.link,
|
||||
date: item.pubDate,
|
||||
content: relatedNews,
|
||||
});
|
||||
console.log(item.title);
|
||||
});
|
||||
return array;
|
||||
} catch (error) {
|
||||
console.error("Error fetching RSS:", error);
|
||||
throw createError({
|
||||
statusCode: 500,
|
||||
message: "Failed to fetch RSS feed",
|
||||
});
|
||||
}
|
||||
});
|
|
@ -1,24 +1,29 @@
|
|||
let cachedWords: content | null = null;
|
||||
let lastFetchTime: number | null = null;
|
||||
const CACHE_DURATION = 1000 * 60 * 60 * 3; // Updates every 3 hours.
|
||||
|
||||
interface content {
|
||||
data: string[];
|
||||
}
|
||||
|
||||
import sql from "~/server/components/postgres";
|
||||
export default defineEventHandler(async (event) => {
|
||||
const currentTime = Date.now();
|
||||
if (
|
||||
cachedWords &&
|
||||
lastFetchTime &&
|
||||
currentTime - lastFetchTime < CACHE_DURATION
|
||||
) {
|
||||
return cachedWords;
|
||||
}
|
||||
const fetchWordsFromGitHub = await fetch(
|
||||
"https://raw.githubusercontent.com/hpware/news-analyze/refs/heads/master/words.json",
|
||||
);
|
||||
cachedWords = await fetchWordsFromGitHub.json();
|
||||
lastFetchTime = currentTime;
|
||||
return cachedWords;
|
||||
return {
|
||||
words: [
|
||||
"尺度太小",
|
||||
"比基尼",
|
||||
"無罩",
|
||||
"脫褲",
|
||||
"裸露",
|
||||
"露豐",
|
||||
"V辣",
|
||||
"激露",
|
||||
"E級曲線",
|
||||
"放0肩",
|
||||
"透視裝",
|
||||
"性侵",
|
||||
"裸照",
|
||||
"性感",
|
||||
"找妹",
|
||||
"肉蹼",
|
||||
"超兇北半球",
|
||||
"大露",
|
||||
"色誘",
|
||||
"死亡",
|
||||
"撩妹",
|
||||
"裸上身",
|
||||
],
|
||||
};
|
||||
});
|
||||
|
|
0
server/api/translate/google.ts
Normal file
0
server/api/translate/google.ts
Normal file
|
@ -1,36 +0,0 @@
|
|||
import sql from "~/server/components/postgres";
|
||||
export default defineEventHandler(async (event) => {
|
||||
// Check user data.
|
||||
const userToken = getCookie(event, "token");
|
||||
if (!userToken) {
|
||||
return {
|
||||
error: "ERR_NOT_ALLOWED",
|
||||
};
|
||||
}
|
||||
const checkUserToken = await sql`
|
||||
select * from usertokens
|
||||
where token=${userToken}
|
||||
`;
|
||||
if (checkUserToken.length === 0) {
|
||||
return {
|
||||
error: "ERR_NOT_ALLOWED",
|
||||
};
|
||||
}
|
||||
// Actual function
|
||||
const body = await readBody(event);
|
||||
const clearBadDataRegex = /[@-_.+a-zA-Z0-9]{2,}/;
|
||||
const requestChange = "groq_api_key";
|
||||
const apiKeyqq = body.value.match(clearBadDataRegex);
|
||||
|
||||
const sqlC = await sql.unsafe(
|
||||
`
|
||||
UPDATE user_other_data SET ${requestChange} = $1
|
||||
WHERE username = $2`,
|
||||
[apiKeyqq[0], checkUserToken[0].username],
|
||||
);
|
||||
return {
|
||||
body: body,
|
||||
data: body.value.match(clearBadDataRegex),
|
||||
sqlC: sqlC,
|
||||
};
|
||||
});
|
|
@ -48,5 +48,6 @@ export default defineEventHandler(async (event) => {
|
|||
current_spot: "KEEP_LOGIN",
|
||||
email: fetchViaSQL[0].email,
|
||||
avatarURL: fetchViaSQL[0].avatarurl,
|
||||
firstName: fetchViaSQL[0].firstName,
|
||||
};
|
||||
});
|
||||
|
|
11
server/components/s3.ts
Normal file
11
server/components/s3.ts
Normal file
|
@ -0,0 +1,11 @@
|
|||
import { S3Client } from "bun";
|
||||
|
||||
const s3config = new S3Client({
|
||||
accessKeyId: process.env.S3_ACCESS_KEY,
|
||||
secretAccessKey: process.env.S3_SECRET_KEY,
|
||||
bucket: process.env.S3_BUCKETNAME,
|
||||
acl: "public-read",
|
||||
endpoint: process.env.S3_ENDPOINT,
|
||||
});
|
||||
|
||||
export default s3config;
|
Loading…
Add table
Add a link
Reference in a new issue