我们将使用t-SNE将嵌入向量的维度从1536降至2。当嵌入向量降至二维后，我们可以在二维散点图中进行可视化。该数据集是在Get_embeddings_from_dataset Notebook中创建的。
1. 降维
我们使用t-SNE降维方法将维度降至2维。
import pandas as pd
from sklearn.manifold import TSNE
import numpy as np
from ast import literal_eval
# Load the embeddings
datafile_path = "data/fine_food_reviews_with_embeddings_1k.csv"
df = pd.read_csv(datafile_path)
# Convert to a list of lists of floats
matrix = np.array(df.embedding.apply(literal_eval).to_list())
# Create a t-SNE model and transform the data
tsne = TSNE(n_components=2, perplexity=15, random_state=42, init='random', learning_rate=200)
vis_dims = tsne.fit_transform(matrix)
vis_dims.shape(1000, 2)
2. 绘制嵌入向量
我们根据每条评论的星级评分为其着色，颜色范围从红色到绿色。
即使在降维到2维的情况下，我们也能观察到良好的数据分离效果。
import matplotlib.pyplot as plt
import matplotlib
import numpy as np
colors = ["red", "darkorange", "gold", "turquoise", "darkgreen"]
x = [x for x,y in vis_dims]
y = [y for x,y in vis_dims]
color_indices = df.Score.values - 1
colormap = matplotlib.colors.ListedColormap(colors)
plt.scatter(x, y, c=color_indices, cmap=colormap, alpha=0.3)
for score in [0,1,2,3,4]:
avg_x = np.array(x)[df.Score-1==score].mean()
avg_y = np.array(y)[df.Score-1==score].mean()
color = colors[score]
plt.scatter(avg_x, avg_y, marker='x', color=color, s=100)
plt.title("Amazon ratings visualized in language using t-SNE")Text(0.5, 1.0, 'Amazon ratings visualized in language using t-SNE')
